-- Logs begin at Fri 2022-08-05 13:30:40 UTC, end at Fri 2022-08-05 14:53:11 UTC. -- Aug 05 14:05:46.584106 dv-cinder-802955-8-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 05 14:05:46.594670 dv-cinder-802955-8-iscsi devstack@keystone.service[80311]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 05 14:05:46.597023 dv-cinder-802955-8-iscsi devstack@keystone.service[80311]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Aug 5 14:05:46 2022] *** Aug 05 14:05:46.597023 dv-cinder-802955-8-iscsi devstack@keystone.service[80311]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 05 14:05:46.597023 dv-cinder-802955-8-iscsi devstack@keystone.service[80311]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 05 14:05:46.597023 dv-cinder-802955-8-iscsi devstack@keystone.service[80311]: nodename: dv-cinder-802955-8-iscsi Aug 05 14:05:46.597023 dv-cinder-802955-8-iscsi devstack@keystone.service[80311]: machine: x86_64 Aug 05 14:05:46.597023 dv-cinder-802955-8-iscsi devstack@keystone.service[80311]: clock source: unix Aug 05 14:05:46.597023 dv-cinder-802955-8-iscsi devstack@keystone.service[80311]: pcre jit disabled Aug 05 14:05:46.597023 dv-cinder-802955-8-iscsi devstack@keystone.service[80311]: detected number of CPU cores: 4 Aug 05 14:05:46.597023 dv-cinder-802955-8-iscsi devstack@keystone.service[80311]: current working directory: / Aug 05 14:05:46.597023 dv-cinder-802955-8-iscsi devstack@keystone.service[80311]: detected binary path: /usr/bin/uwsgi-core Aug 05 14:05:46.597593 dv-cinder-802955-8-iscsi devstack@keystone.service[80311]: your processes number limit is 47680 Aug 05 14:05:46.597593 dv-cinder-802955-8-iscsi devstack@keystone.service[80311]: your memory page size is 4096 bytes Aug 05 14:05:46.597593 dv-cinder-802955-8-iscsi devstack@keystone.service[80311]: detected max file descriptor number: 2048 Aug 05 14:05:46.597593 dv-cinder-802955-8-iscsi devstack@keystone.service[80311]: lock engine: pthread robust mutexes Aug 05 14:05:46.597593 dv-cinder-802955-8-iscsi devstack@keystone.service[80311]: thunder lock: enabled Aug 05 14:05:46.597593 dv-cinder-802955-8-iscsi devstack@keystone.service[80311]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 05 14:05:46.597593 dv-cinder-802955-8-iscsi devstack@keystone.service[80311]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 05 14:05:46.637348 dv-cinder-802955-8-iscsi devstack@keystone.service[80311]: Python main interpreter initialized at 0x55ad6b8479c0 Aug 05 14:05:46.637348 dv-cinder-802955-8-iscsi devstack@keystone.service[80311]: python threads support enabled Aug 05 14:05:46.637348 dv-cinder-802955-8-iscsi devstack@keystone.service[80311]: your server socket listen backlog is limited to 100 connections Aug 05 14:05:46.637348 dv-cinder-802955-8-iscsi devstack@keystone.service[80311]: your mercy for graceful operations on workers is 90 seconds Aug 05 14:05:46.637679 dv-cinder-802955-8-iscsi devstack@keystone.service[80311]: mapped 403077 bytes (393 KB) for 2 cores Aug 05 14:05:46.637756 dv-cinder-802955-8-iscsi devstack@keystone.service[80311]: *** Operational MODE: preforking *** Aug 05 14:05:46.638006 dv-cinder-802955-8-iscsi devstack@keystone.service[80311]: *** uWSGI is running in multiple interpreter mode *** Aug 05 14:05:46.638006 dv-cinder-802955-8-iscsi devstack@keystone.service[80311]: spawned uWSGI master process (pid: 80311) Aug 05 14:05:46.638199 dv-cinder-802955-8-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 05 14:05:46.639866 dv-cinder-802955-8-iscsi devstack@keystone.service[80311]: spawned uWSGI worker 1 (pid: 80321, cores: 1) Aug 05 14:05:46.639866 dv-cinder-802955-8-iscsi devstack@keystone.service[80311]: spawned uWSGI worker 2 (pid: 80322, cores: 1) Aug 05 14:05:46.639866 dv-cinder-802955-8-iscsi devstack@keystone.service[80311]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 05 14:05:52.217472 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 05 14:05:52.217472 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: from cryptography.utils import int_from_bytes Aug 05 14:05:52.218270 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 05 14:05:52.218270 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: from cryptography.utils import int_from_bytes Aug 05 14:05:52.221055 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 05 14:05:52.221055 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: from cryptography.utils import int_from_bytes Aug 05 14:05:52.221588 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 05 14:05:52.221588 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: from cryptography.utils import int_from_bytes Aug 05 14:05:52.268698 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 05 14:05:52.268874 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 05 14:05:52.269067 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] command line args: [] {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 05 14:05:52.269336 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 05 14:05:52.269513 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ================================================================================ {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 05 14:05:52.269764 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] admin_token = **** {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.270014 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] config_dir = [] {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.270199 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.270379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] config_source = [] {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.270557 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] debug = True {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.270756 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.271251 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.271346 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.271539 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.271786 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.272014 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] list_limit = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.272194 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] log_config_append = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.272370 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.272614 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] log_dir = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.272782 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] log_file = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.273016 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.273262 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.273648 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.273898 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.274077 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.274253 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.274496 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.274740 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.274974 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.275207 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.275442 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.275674 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.275918 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.276160 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.276448 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.276953 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.277132 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] publish_errors = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.277362 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.277590 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.277767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.278011 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.278242 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.278475 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] standard_threads = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.278706 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.278948 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.279124 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.279299 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] use_journal = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.279474 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] use_json = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.279648 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] use_stderr = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.279847 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] use_syslog = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.280057 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.280312 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.280575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.280827 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 05 14:05:52.280927 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.281027 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 05 14:05:52.281127 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.281201 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] command line args: [] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 05 14:05:52.281282 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.281470 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 05 14:05:52.281558 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.281637 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ================================================================================ {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 05 14:05:52.281746 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.281879 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] admin_token = **** {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.281981 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.282125 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] config_dir = [] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.282226 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.282311 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.282471 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.282556 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] config_source = [] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.282629 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] debug = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.282743 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.282825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.282920 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.283036 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.283168 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.283302 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.283404 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.283490 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.283648 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.283732 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.283893 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.283984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] list_limit = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.284064 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] log_config_append = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.284141 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.284213 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.284371 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.284509 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] log_dir = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.284586 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] log_file = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.284664 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.284811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.284896 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.285035 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.285115 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.285279 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.285363 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.285522 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.285602 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.285684 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.285826 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.285911 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.286074 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.286153 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.286307 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.286459 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.286541 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.286612 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.286749 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.286834 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.286977 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.287064 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.287206 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.287291 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.287446 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.287525 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.287673 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.287752 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.287950 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.288041 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.288200 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.289113 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.300762 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.300845 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] publish_errors = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.300907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.300966 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.301033 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.301094 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.301159 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.301224 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.301285 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.301348 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.301417 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.301477 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.301536 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.301600 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] standard_threads = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.301671 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.301733 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.301799 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.301859 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.301859 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.301963 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.302027 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] use_journal = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.302093 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.302162 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] use_json = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.302162 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] use_stderr = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.302267 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.302330 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] use_syslog = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.302389 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.302447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 14:05:52.302600 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.302666 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.302725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.302784 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.302848 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.302920 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.302989 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.303051 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.303109 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.303192 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.303253 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.303317 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.303384 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.303450 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.303515 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.303575 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.303633 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.303692 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.303757 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.303817 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.303934 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.304545 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.304663 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.304745 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.304816 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.304878 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.304938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.304996 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.305054 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.305112 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.305179 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.305246 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.305316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.305377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.305477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.305477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.305610 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.305687 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.305747 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.305805 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.305864 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.305864 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.305975 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.305975 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.306093 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.306152 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.306215 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.306275 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.306341 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.306401 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.306469 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.306531 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.306594 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.306654 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.306712 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.306780 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.306841 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.306899 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.306958 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.307023 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.307083 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.307149 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.307210 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.307268 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.307327 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.307391 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.307450 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.307516 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.307582 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.307655 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.307714 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.307773 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.307831 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.307898 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.307958 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.307958 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.308074 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.308134 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.308192 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.308261 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.308322 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.308386 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.308472 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.308535 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.308603 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.308673 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.308735 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.308798 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.308858 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.308925 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.308990 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.309058 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.309118 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.309177 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.309235 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.309300 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.309359 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.309431 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.309492 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.309560 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.309627 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.309709 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.309769 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.309901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.309964 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.310023 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.310023 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.310192 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.310252 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.310326 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.310399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.310460 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.310525 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.310593 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.310653 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.310712 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] auth.external = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.310777 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.310838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.310896 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.310896 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.310999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.311058 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.311124 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.311185 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.311244 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] auth.password = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.311312 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.311374 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] auth.token = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.311432 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.311498 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.311558 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.311628 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.311689 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.311748 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.311823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.311890 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.311951 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.312011 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.312069 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.312134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.312134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.312253 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.312314 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] credential.caching = True {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.312373 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.312673 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.312747 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.312747 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.312871 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.312934 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.312993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.313052 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.313110 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.313168 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.313233 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.313295 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.313354 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.313412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.313470 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.313529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.313595 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.313655 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.313714 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.313773 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.313831 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.313894 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.313961 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.314022 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.314154 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.314278 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.314386 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.314485 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.314485 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.314691 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.314802 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.314902 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.314999 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.315108 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] federation.caching = True {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.315205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.315309 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.315416 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.315526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.315624 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.315728 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.315826 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.315938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.316031 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.316153 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.316232 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.316350 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.316447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.316524 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.316657 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.316738 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.316857 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.316933 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.317007 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.317125 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.317210 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.317287 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.317366 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.317441 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] identity.caching = True {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.317558 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.317642 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.317725 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.317801 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.317920 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.317997 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.318070 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] auth.external = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.318235 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.318320 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.318458 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.318531 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.318594 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.318654 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.318839 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.318930 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] auth.password = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.319062 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.319143 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] auth.token = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.319253 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.319358 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.319448 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.319582 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.319762 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.319914 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.320055 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.320150 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.320242 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.320319 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.320459 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.320582 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.320666 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.320741 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.320741 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] credential.caching = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.320938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.321017 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.321091 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.321164 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.321284 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.321375 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.321451 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.321524 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.321646 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.321724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.321798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.321887 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.321969 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.321969 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.322110 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.322182 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.322255 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.322383 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.322463 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.322537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.322610 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.322683 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.322755 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.322836 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.322911 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.322984 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.323057 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.323130 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.323203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.323284 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.323359 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.323432 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.323505 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] federation.caching = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.323577 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.323649 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.323783 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.323863 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.323943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.324018 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.324090 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.324163 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.324243 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.324319 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.324392 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.324543 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.324628 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.324704 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.324833 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.324919 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.325069 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.325152 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.325302 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.325382 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.325547 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.325631 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.325776 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.325856 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] identity.caching = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.326006 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.326099 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.326248 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.326328 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.326485 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.326601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.326720 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.326832 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.326963 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.327046 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.327190 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.327272 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.327420 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.user = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.327508 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.327660 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.327742 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.327906 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.328012 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.328136 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.328252 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.328353 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.328476 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.328610 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.328717 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.328852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.329466 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.329466 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.329806 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.329925 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.330041 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.330041 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.330268 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.330268 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.330490 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.330615 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.330732 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.330849 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.330973 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.331091 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.331214 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.331333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.331449 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.331566 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.331690 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.331690 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.331911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.332027 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.332143 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.332258 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.332383 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.332525 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.332642 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.332760 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.332919 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.333040 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.333170 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.333289 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.333406 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.333524 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.333640 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.333756 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.333880 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.334029 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.334157 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.334274 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.334391 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.334509 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.334634 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.334752 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.334868 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.334984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.334984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.335316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.335424 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.335525 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.335681 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.335847 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.335969 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.336087 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.336245 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.336368 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.336534 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.336654 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.336770 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.336887 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.337017 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.337137 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.337253 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.337369 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] resource.caching = True {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.337485 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.337600 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.337731 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.337851 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.337967 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.338074 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.338172 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.338172 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.338434 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.338543 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.338693 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.338836 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.338936 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.339052 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.339178 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.339296 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.339412 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.339528 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.339643 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.339768 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.339768 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.339988 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] role.caching = True {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.340104 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.340221 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] role.driver = sql {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.340336 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.340585 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.340708 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.340828 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.340945 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.341107 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.341220 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.341344 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.341463 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.341580 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.341696 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.341696 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.341914 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.342037 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.342156 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.342272 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.342388 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.342532 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.342650 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.342775 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.342899 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.343015 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.343131 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.343247 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.343363 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.343488 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.343606 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.343723 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.343838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.343956 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.343956 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.344180 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.344298 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.344444 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.344592 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.344711 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.344828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.344958 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.345078 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.345194 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.345309 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.345425 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.345540 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.345665 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.345782 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.345898 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.346015 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.346131 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.346247 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.346372 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.346489 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.346606 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.346722 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.346864 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.346981 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.347172 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.347290 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] resource.caching = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.347407 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.347523 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.347639 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.347756 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.347881 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.347999 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.348117 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.348233 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.348356 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.348507 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.348660 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.348781 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.348899 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] token.caching = True {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.349016 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.349132 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.349257 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.349377 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.349493 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.349609 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.349725 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] role.caching = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.349841 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.349966 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] role.driver = sql {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.350084 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.350200 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.350318 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.350433 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.350549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.350673 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.351301 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.351420 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.351537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.351653 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.351769 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.351895 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.352013 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.352130 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.352246 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.352363 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.352531 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.352693 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.352822 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.352940 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.353057 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.353174 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.353290 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.353416 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.353416 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.353637 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.353753 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.353870 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] database.connection = **** {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.353870 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.354105 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.354231 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.354349 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.354467 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.354582 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.354699 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.354815 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.354938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.355063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.355180 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.355296 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.355413 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.355530 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.355654 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.355772 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.355889 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.356005 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.356121 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.356237 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.356361 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.356495 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.356614 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.356730 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.356880 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.356997 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.357126 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.357235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.357350 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.357466 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.357583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.357698 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.357823 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.357941 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.358057 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.358174 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.358290 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80322) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 05 14:05:52.358406 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.358406 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] token.caching = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.358406 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.358406 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.358406 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.358406 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.358406 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.358406 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.359247 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.359247 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.359247 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.359247 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.359663 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.359663 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.359888 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.360002 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.360218 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.360477 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.360770 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.361028 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.connection = **** {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.361276 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.361519 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.361753 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.361989 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.362221 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.362453 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.362684 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.362914 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.363156 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.363389 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.363629 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.363860 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.364104 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.364333 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.364597 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.364824 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.365060 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.365290 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 14:05:52.365458 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 05 14:05:52.386717 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80322) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.386950 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80322) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.387175 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80322) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.390852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.391083 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.391303 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.395541 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80322) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.399779 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.425998 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80322) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.426229 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80322) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.427643 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.427873 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.481460 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80322) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.481745 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.532993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80322) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.533229 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80322) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.537168 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.537401 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.605964 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.606189 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.606410 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.606628 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.606711 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80322) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.606844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.606927 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80322) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.607056 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.607142 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80322) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.607353 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80322) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.607571 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80322) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.607783 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80322) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.630625 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.630815 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80322) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.630944 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.631030 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80322) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.631108 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.631258 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.631346 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80322) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.631473 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.631559 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80322) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.631683 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.631778 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80322) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.631891 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80322) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.631975 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.632152 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80322) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.643008 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.643305 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80322) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.656041 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.656275 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.656523 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.656731 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80322) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.656816 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.656945 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.657030 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80322) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.657157 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.657242 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80322) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.657368 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.657462 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80322) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.657589 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80322) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.657802 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80322) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.658013 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80322) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 14:05:52.674694 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.674899 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 05 14:05:52.675230 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.675427 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 05 14:05:52.675618 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.675703 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.675828 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 05 14:05:52.675920 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.676164 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.676248 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.676365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 05 14:05:52.676473 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.676666 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.676928 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.677182 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.677434 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.686210 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.686481 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.686744 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.687015 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.687335 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.687609 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.687874 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.688152 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.694002 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80321) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 14:05:52.695130 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80322) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 14:05:52.699018 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80321) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 14:05:52.699294 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.699570 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.699848 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.699978 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80322) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 14:05:52.700125 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.700261 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.700379 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.700566 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.700652 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.700844 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.700928 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.701122 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.701213 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.701380 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.701463 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.701647 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.701730 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.701917 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.702182 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.702447 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.702708 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.725099 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.726711 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.727274 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80321) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 14:05:52.727550 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.728353 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80322) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 14:05:52.728659 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.732201 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80321) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 14:05:52.732453 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.732734 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.733283 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80322) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 14:05:52.733561 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.733842 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.739273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80321) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 14:05:52.739551 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.740468 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80322) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 14:05:52.740747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.744339 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80321) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 14:05:52.744650 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.744926 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.745199 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.745484 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.745566 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80322) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 14:05:52.745761 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.745853 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.746033 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.746118 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.746291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.746376 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.746595 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.746876 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.747140 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.747408 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.763252 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.763526 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.764313 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.764621 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.767432 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80321) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 14:05:52.768531 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80322) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 14:05:52.770586 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.770875 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.771676 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.771970 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.776085 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80321) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 14:05:52.776968 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80322) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 14:05:52.779521 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80321) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 14:05:52.780705 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80322) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 14:05:52.782703 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.782996 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.783275 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.783552 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.783822 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.783906 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.784093 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.784178 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.784445 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.784738 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.785016 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.785293 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.800622 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80321) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 14:05:52.800896 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.801158 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.801415 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.801774 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80322) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 14:05:52.802053 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.802317 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.802576 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.809178 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.809467 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.810713 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.810993 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.811076 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.811276 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.813803 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80321) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 14:05:52.814092 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.814176 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80322) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 14:05:52.814376 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.814463 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.814650 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.814822 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.814909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.815107 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.815383 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.826756 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80321) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 14:05:52.826894 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80322) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 14:05:52.827029 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.827169 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.827294 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.827445 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.827559 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.827721 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.827835 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.827999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.828102 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.828270 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.828375 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.828641 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.844605 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80321) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 14:05:52.844735 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80322) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 14:05:52.847541 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80321) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 14:05:52.847686 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80322) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 14:05:52.850482 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.850647 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.853658 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.853756 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.855149 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80321) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 14:05:52.855233 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80322) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 14:05:52.855420 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.855504 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.855705 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.855787 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.862136 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.862222 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.863732 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80321) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 14:05:52.863817 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80322) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 14:05:52.866707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.866798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.866990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.867155 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.867307 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.867385 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.867526 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.867618 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.875778 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80321) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 14:05:52.875871 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80322) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 14:05:52.876067 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.876150 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.876354 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.876476 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.883747 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80321) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 14:05:52.883903 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80322) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 14:05:52.884015 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.884181 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.884277 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.884483 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.884571 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.884758 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.884841 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.885025 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.885110 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.885294 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.885375 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.885561 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.885643 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.885843 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.885958 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.886230 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.886312 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.886494 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.886582 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.886760 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.886843 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.887025 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.887106 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.887276 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.887367 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.887533 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80322) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 14:05:52.921548 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55ad6b8479c0 pid: 80321 (default app) Aug 05 14:05:52.922004 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55ad6b8479c0 pid: 80322 (default app) Aug 05 14:05:52.924816 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f831549-4e3b-4f1f-b007-26eba4479fc4 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:05:52.925101 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f831549-4e3b-4f1f-b007-26eba4479fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:05:52.925371 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f831549-4e3b-4f1f-b007-26eba4479fc4 None None] PATH_INFO: `/v3` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:05:52.930467 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1/1] 10.222.0.26 () {50 vars in 791 bytes} [Fri Aug 5 14:05:52 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:01.533819 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f4dac-540a-4947-817d-800b45a10af0 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:01.534281 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f4dac-540a-4947-817d-800b45a10af0 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:01.534374 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f4dac-540a-4947-817d-800b45a10af0 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:01.538884 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1/2] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:01.544240 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e25977-2112-48e0-96d4-84b1455c5d42 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:01.544692 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e25977-2112-48e0-96d4-84b1455c5d42 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:01.544814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e25977-2112-48e0-96d4-84b1455c5d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:01.614691 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: INFO dbcounter [None req-22e25977-2112-48e0-96d4-84b1455c5d42 None None] Registered counter for database keystone Aug 05 14:06:01.634892 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG oslo_db.sqlalchemy.engines [None req-22e25977-2112-48e0-96d4-84b1455c5d42 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80321) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 05 14:06:01.641057 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writer thread running {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 05 14:06:01.819341 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG passlib.handlers.bcrypt [None req-22e25977-2112-48e0-96d4-84b1455c5d42 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80321) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 05 14:06:01.820251 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG passlib.handlers.bcrypt [None req-22e25977-2112-48e0-96d4-84b1455c5d42 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80321) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 05 14:06:01.873931 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-22e25977-2112-48e0-96d4-84b1455c5d42 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:01.875796 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-22e25977-2112-48e0-96d4-84b1455c5d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:02.006745 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 2/3] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:01 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 05 14:06:02.144769 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f4dac-540a-4947-817d-800b45a10af0 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:02.214460 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: INFO dbcounter [None req-776f4dac-540a-4947-817d-800b45a10af0 None None] Registered counter for database keystone Aug 05 14:06:02.236105 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG oslo_db.sqlalchemy.engines [None req-776f4dac-540a-4947-817d-800b45a10af0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80322) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 05 14:06:02.242870 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writer thread running {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 05 14:06:02.363241 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f4dac-540a-4947-817d-800b45a10af0 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:02.367693 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd1bb5e-158a-4405-ad1e-1dfdcee4103a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:02.368331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1bb5e-158a-4405-ad1e-1dfdcee4103a admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:02.368640 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1bb5e-158a-4405-ad1e-1dfdcee4103a admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:02.368915 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1bb5e-158a-4405-ad1e-1dfdcee4103a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:02.377234 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd1bb5e-158a-4405-ad1e-1dfdcee4103a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:02.429879 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ccd1bb5e-158a-4405-ad1e-1dfdcee4103a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 05 14:06:02.430182 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd1bb5e-158a-4405-ad1e-1dfdcee4103a admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:02.439859 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 2/4] 10.222.0.26 () {58 vars in 1324 bytes} [Fri Aug 5 14:06:02 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 05 14:06:02.445783 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e25977-2112-48e0-96d4-84b1455c5d42 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:02.459959 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e25977-2112-48e0-96d4-84b1455c5d42 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:02.465075 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7183df-1a4c-45c7-a556-28094f3f8250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:02.466176 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7183df-1a4c-45c7-a556-28094f3f8250 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:02.466461 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7183df-1a4c-45c7-a556-28094f3f8250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:02.466735 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7183df-1a4c-45c7-a556-28094f3f8250 admin admin] PATH_INFO: `/v3/projects/b40e0244e98a49699194b094ad410d1e` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:02.468223 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7183df-1a4c-45c7-a556-28094f3f8250 admin admin] RBAC: Authorizing `identity:get_project(project_id=b40e0244e98a49699194b094ad410d1e)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:02.518510 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: WARNING keystone.common.rbac_enforcer.enforcer [None req-bf7183df-1a4c-45c7-a556-28094f3f8250 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 05 14:06:02.518854 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7183df-1a4c-45c7-a556-28094f3f8250 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:02.524038 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 3/5] 10.222.0.26 () {56 vars in 1207 bytes} [Fri Aug 5 14:06:02 2022] GET /identity/v3/projects/b40e0244e98a49699194b094ad410d1e => generated 344 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:06:04.495886 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88872c2-bd5c-403f-a612-5aff325e786c None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:04.496477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88872c2-bd5c-403f-a612-5aff325e786c None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:04.496477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88872c2-bd5c-403f-a612-5aff325e786c None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:04.500856 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 3/6] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:04.506164 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455f192-4b25-4e05-ac82-c1128c554f0b None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:04.506536 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455f192-4b25-4e05-ac82-c1128c554f0b None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:04.506713 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455f192-4b25-4e05-ac82-c1128c554f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:04.526985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-2455f192-4b25-4e05-ac82-c1128c554f0b None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:04.528503 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-2455f192-4b25-4e05-ac82-c1128c554f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:04.536258 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 4/7] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:04 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 05 14:06:04.558988 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88872c2-bd5c-403f-a612-5aff325e786c None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:04.572067 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88872c2-bd5c-403f-a612-5aff325e786c None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:04.576364 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9d013-0f8e-461c-9e24-a787b99d5df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:04.576992 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9d013-0f8e-461c-9e24-a787b99d5df6 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:04.577269 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9d013-0f8e-461c-9e24-a787b99d5df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:04.577541 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9d013-0f8e-461c-9e24-a787b99d5df6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:04.585417 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb9d013-0f8e-461c-9e24-a787b99d5df6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:04.587420 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb9d013-0f8e-461c-9e24-a787b99d5df6 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:04.596029 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 4/8] 10.222.0.26 () {58 vars in 1324 bytes} [Fri Aug 5 14:06:04 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 05 14:06:04.601424 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455f192-4b25-4e05-ac82-c1128c554f0b None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:04.610385 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455f192-4b25-4e05-ac82-c1128c554f0b None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:04.614757 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448bc388-a487-41be-9cfd-bda14c83b9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:04.615516 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448bc388-a487-41be-9cfd-bda14c83b9c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:04.615798 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448bc388-a487-41be-9cfd-bda14c83b9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:04.616075 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448bc388-a487-41be-9cfd-bda14c83b9c7 admin admin] PATH_INFO: `/v3/users/5e3fbbd9c18b4be39ebc01f40d1df40f` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:04.618068 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448bc388-a487-41be-9cfd-bda14c83b9c7 admin admin] RBAC: Authorizing `identity:get_user(user_id=5e3fbbd9c18b4be39ebc01f40d1df40f)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:04.619870 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448bc388-a487-41be-9cfd-bda14c83b9c7 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:04.621478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 5/9] 10.222.0.26 () {56 vars in 1198 bytes} [Fri Aug 5 14:06:04 2022] GET /identity/v3/users/5e3fbbd9c18b4be39ebc01f40d1df40f => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:06.656833 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affa512-4a23-4c5c-87c7-0515c98d8c75 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:06.657366 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affa512-4a23-4c5c-87c7-0515c98d8c75 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:06.657366 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affa512-4a23-4c5c-87c7-0515c98d8c75 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:06.658709 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 5/10] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:06.663858 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0767ed-83ca-4640-bb63-8228d1685e11 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:06.664293 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0767ed-83ca-4640-bb63-8228d1685e11 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:06.664387 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0767ed-83ca-4640-bb63-8228d1685e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:06.683033 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-cb0767ed-83ca-4640-bb63-8228d1685e11 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:06.684203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-cb0767ed-83ca-4640-bb63-8228d1685e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:06.701347 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3495a-022f-4cfc-8a05-7891c31e7333 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:06.701629 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3495a-022f-4cfc-8a05-7891c31e7333 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:06.701905 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3495a-022f-4cfc-8a05-7891c31e7333 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:06.703222 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 6/11] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:06:06.708279 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f1dc1-9309-4e31-8c2c-543dca6cf449 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:06.708594 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f1dc1-9309-4e31-8c2c-543dca6cf449 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:06.708874 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f1dc1-9309-4e31-8c2c-543dca6cf449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:06.751700 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG passlib.handlers.bcrypt [None req-993f1dc1-9309-4e31-8c2c-543dca6cf449 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80322) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 05 14:06:06.752118 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG passlib.handlers.bcrypt [None req-993f1dc1-9309-4e31-8c2c-543dca6cf449 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80322) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 05 14:06:06.764548 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 6/12] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:06.787034 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0767ed-83ca-4640-bb63-8228d1685e11 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:06.797533 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-993f1dc1-9309-4e31-8c2c-543dca6cf449 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:06.799120 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-993f1dc1-9309-4e31-8c2c-543dca6cf449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:06.811218 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:06.890102 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0767ed-83ca-4640-bb63-8228d1685e11 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:06.900781 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 7/13] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 14:06:06.922168 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993f1dc1-9309-4e31-8c2c-543dca6cf449 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:06.927770 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d852e5d-009f-465f-b4b8-529cfcc1df67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:06.928437 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d852e5d-009f-465f-b4b8-529cfcc1df67 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:06.929281 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d852e5d-009f-465f-b4b8-529cfcc1df67 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:06.929584 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d852e5d-009f-465f-b4b8-529cfcc1df67 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:06.940667 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d852e5d-009f-465f-b4b8-529cfcc1df67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:06.950412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-8d852e5d-009f-465f-b4b8-529cfcc1df67 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:06:06.950412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:06.950412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:06.950412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:06.950412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:06.950412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:06.950412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:06.950412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:06.950412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:06.950412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:06.950412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:06.950412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:06.950412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:06.950412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:06.950412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:06.950412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:06.951409 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:06.951409 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:06.951409 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:06.951409 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:06.951409 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:06.951409 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:06.951409 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:06.951409 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:06.951409 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:06.951409 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:06.951409 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:06.951409 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:06.951409 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:06.951409 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:06.951409 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:06.951409 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:06.952243 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:06.952243 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:06.952243 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:06.952243 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:06.952243 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:06.952243 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:06.952243 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:06.952243 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:06.952243 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:06.952243 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:06.952243 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:06:06.952243 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:06:06.952243 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 7/14] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Aug 5 14:06:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:06:06.957312 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d852e5d-009f-465f-b4b8-529cfcc1df67 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:07.022756 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993f1dc1-9309-4e31-8c2c-543dca6cf449 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:07.025595 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:07.039455 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:07.057512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d852e5d-009f-465f-b4b8-529cfcc1df67 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:07.059232 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa548c-c8cf-4e02-a63f-bb4d468ac770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:07.060097 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa548c-c8cf-4e02-a63f-bb4d468ac770 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:07.060372 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa548c-c8cf-4e02-a63f-bb4d468ac770 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:07.060670 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa548c-c8cf-4e02-a63f-bb4d468ac770 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:07.090755 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd66ad5-fe42-4aa8-b0e7-ec520400d3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:07.091363 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd66ad5-fe42-4aa8-b0e7-ec520400d3e6 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:07.091638 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd66ad5-fe42-4aa8-b0e7-ec520400d3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:07.091910 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd66ad5-fe42-4aa8-b0e7-ec520400d3e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:07.092652 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd66ad5-fe42-4aa8-b0e7-ec520400d3e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:07.092956 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd66ad5-fe42-4aa8-b0e7-ec520400d3e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:07.094342 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd66ad5-fe42-4aa8-b0e7-ec520400d3e6 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:07.107671 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 8/15] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:06:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:06:07.113158 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd66ad5-fe42-4aa8-b0e7-ec520400d3e6 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:07.126966 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aa548c-c8cf-4e02-a63f-bb4d468ac770 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:07.128509 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aa548c-c8cf-4e02-a63f-bb4d468ac770 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:07.233262 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 8/16] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:06:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 14:06:07.239191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa548c-c8cf-4e02-a63f-bb4d468ac770 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:07.242946 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:07.319353 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd66ad5-fe42-4aa8-b0e7-ec520400d3e6 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:07.337587 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa548c-c8cf-4e02-a63f-bb4d468ac770 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:07.352715 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775877a-8d90-4b33-bb9d-1d305f57bc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:07.353354 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775877a-8d90-4b33-bb9d-1d305f57bc7b None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:07.353630 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775877a-8d90-4b33-bb9d-1d305f57bc7b None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:07.354049 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775877a-8d90-4b33-bb9d-1d305f57bc7b None admin] PATH_INFO: `/v3/users/5e3fbbd9c18b4be39ebc01f40d1df40f` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:07.355047 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4775877a-8d90-4b33-bb9d-1d305f57bc7b None admin] RBAC: Authorizing `identity:get_user(user_id=5e3fbbd9c18b4be39ebc01f40d1df40f)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:07.356510 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4775877a-8d90-4b33-bb9d-1d305f57bc7b None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:07.358103 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 9/17] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:06:07 2022] GET /identity/v3/users/5e3fbbd9c18b4be39ebc01f40d1df40f => generated 247 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:06:07.363500 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775877a-8d90-4b33-bb9d-1d305f57bc7b None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:07.372862 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb9d25-8553-4b7f-b079-49944bfa67bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:07.373723 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb9d25-8553-4b7f-b079-49944bfa67bc None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:07.374254 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb9d25-8553-4b7f-b079-49944bfa67bc None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:07.374526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb9d25-8553-4b7f-b079-49944bfa67bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:07.378849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:07.392501 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb9d25-8553-4b7f-b079-49944bfa67bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:07.393958 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb9d25-8553-4b7f-b079-49944bfa67bc None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:07.411795 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-65bb9d25-8553-4b7f-b079-49944bfa67bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:06:07.411795 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:07.411795 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:07.411795 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:07.411795 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:07.411795 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:07.411795 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:07.411795 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:07.411795 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:07.411795 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:07.411795 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:07.411795 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:07.411795 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:06:07.411795 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:06:07.411795 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:06:07.412733 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:06:07.412733 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:07.412733 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:07.412733 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:07.412733 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:07.412733 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:07.412733 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:07.412733 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:07.412733 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:07.412733 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:07.412733 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:07.412733 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:07.412733 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:07.412733 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:07.412733 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:07.412733 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:07.412733 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:07.414094 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:06:07.414094 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:06:07.414094 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:06:07.414094 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:06:07.414094 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 9/18] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:06:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:06:07.419734 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb9d25-8553-4b7f-b079-49944bfa67bc None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:07.463527 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775877a-8d90-4b33-bb9d-1d305f57bc7b None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:07.475120 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:07.497331 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55524541-45e6-45a5-b236-56a05ca76d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:07.497964 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55524541-45e6-45a5-b236-56a05ca76d22 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:07.498237 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55524541-45e6-45a5-b236-56a05ca76d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:07.498503 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55524541-45e6-45a5-b236-56a05ca76d22 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:07.499906 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55524541-45e6-45a5-b236-56a05ca76d22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:07.501374 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55524541-45e6-45a5-b236-56a05ca76d22 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:07.502914 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 10/19] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:06:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:06:07.509111 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55524541-45e6-45a5-b236-56a05ca76d22 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:07.518169 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb9d25-8553-4b7f-b079-49944bfa67bc None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:07.552987 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed4d7af-4b6e-4a08-9071-9c261550f23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:07.553633 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed4d7af-4b6e-4a08-9071-9c261550f23f None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:07.553922 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed4d7af-4b6e-4a08-9071-9c261550f23f None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:07.554185 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed4d7af-4b6e-4a08-9071-9c261550f23f None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:07.554878 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed4d7af-4b6e-4a08-9071-9c261550f23f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:07.555179 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed4d7af-4b6e-4a08-9071-9c261550f23f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:07.556589 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed4d7af-4b6e-4a08-9071-9c261550f23f None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:07.573843 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 10/20] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:06:07.594936 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:07.615613 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55524541-45e6-45a5-b236-56a05ca76d22 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:07.651968 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded6d81b-ff5f-4538-9b7a-cee2a4c19aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:07.652630 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded6d81b-ff5f-4538-9b7a-cee2a4c19aec None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:07.652906 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded6d81b-ff5f-4538-9b7a-cee2a4c19aec None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:07.653173 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded6d81b-ff5f-4538-9b7a-cee2a4c19aec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:07.653983 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded6d81b-ff5f-4538-9b7a-cee2a4c19aec None admin] RBAC: Adding query filter params (role.id=299c794c7acb4a5ca36288c3191aad10, scope.domain.id=default, user.id=5e3fbbd9c18b4be39ebc01f40d1df40f) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:07.654287 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded6d81b-ff5f-4538-9b7a-cee2a4c19aec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:07.655771 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded6d81b-ff5f-4538-9b7a-cee2a4c19aec None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:07.663914 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 11/21] 10.222.0.26 () {56 vars in 1322 bytes} [Fri Aug 5 14:06:07 2022] GET /identity/v3/role_assignments?user.id=5e3fbbd9c18b4be39ebc01f40d1df40f&scope.domain.id=default&role.id=299c794c7acb4a5ca36288c3191aad10 => generated 233 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:06:10.103345 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe188d-6e7c-4430-8c94-89ede47c0f15 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:10.104473 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe188d-6e7c-4430-8c94-89ede47c0f15 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:10.104964 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe188d-6e7c-4430-8c94-89ede47c0f15 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:10.106793 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 11/22] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:10.111736 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f311f-6eef-428e-90bb-fb89f2e936be None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:10.112164 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f311f-6eef-428e-90bb-fb89f2e936be None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:10.112355 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f311f-6eef-428e-90bb-fb89f2e936be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:10.141413 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-245f311f-6eef-428e-90bb-fb89f2e936be None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:10.142618 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-245f311f-6eef-428e-90bb-fb89f2e936be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:10.158436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed85447-a7c2-4b82-bd2b-a8005177f499 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:10.159053 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed85447-a7c2-4b82-bd2b-a8005177f499 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:10.159534 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed85447-a7c2-4b82-bd2b-a8005177f499 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:10.161098 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 12/23] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:10.166520 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe95e1-30ed-498f-a80e-10a80eaf4503 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:10.167176 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe95e1-30ed-498f-a80e-10a80eaf4503 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:10.167669 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe95e1-30ed-498f-a80e-10a80eaf4503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:10.197219 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-edfe95e1-30ed-498f-a80e-10a80eaf4503 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:10.199123 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-edfe95e1-30ed-498f-a80e-10a80eaf4503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:10.278664 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 12/24] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 14:06:10.280297 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558184d-7a99-4324-81d8-11310a52b612 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:10.280817 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558184d-7a99-4324-81d8-11310a52b612 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:10.281320 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558184d-7a99-4324-81d8-11310a52b612 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:10.282781 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 13/25] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:10.288950 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4fcb4-b622-4d0a-84f1-01f26098e815 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:10.289471 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4fcb4-b622-4d0a-84f1-01f26098e815 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:10.289944 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4fcb4-b622-4d0a-84f1-01f26098e815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:10.321897 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-dbd4fcb4-b622-4d0a-84f1-01f26098e815 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:10.323641 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-dbd4fcb4-b622-4d0a-84f1-01f26098e815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:10.345443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 13/26] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:10.346297 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe95e1-30ed-498f-a80e-10a80eaf4503 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:10.399805 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 14/27] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:10.405259 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd4fcb4-b622-4d0a-84f1-01f26098e815 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:10.409373 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:10.449697 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:10.537116 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe95e1-30ed-498f-a80e-10a80eaf4503 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:10.543440 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd4fcb4-b622-4d0a-84f1-01f26098e815 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:10.586599 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2460614-1a93-4fae-9daa-d1bdfc873cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:10.587391 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2460614-1a93-4fae-9daa-d1bdfc873cbf None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:10.587725 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df70484-8498-4120-8e1a-12d741a3384b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:10.587995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2460614-1a93-4fae-9daa-d1bdfc873cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:10.588362 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2460614-1a93-4fae-9daa-d1bdfc873cbf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:10.588860 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df70484-8498-4120-8e1a-12d741a3384b None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:10.589280 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df70484-8498-4120-8e1a-12d741a3384b None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:10.589703 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df70484-8498-4120-8e1a-12d741a3384b None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:10.590569 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df70484-8498-4120-8e1a-12d741a3384b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:10.592278 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df70484-8498-4120-8e1a-12d741a3384b None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:10.602360 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2460614-1a93-4fae-9daa-d1bdfc873cbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:10.609095 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-c2460614-1a93-4fae-9daa-d1bdfc873cbf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:06:10.609095 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:10.609095 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:10.609095 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:10.609095 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:10.609095 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:10.609095 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:10.609095 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:10.609095 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:10.609095 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:10.609095 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:10.609095 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:10.609095 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:10.609095 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:10.609095 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:10.609095 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:10.617393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:10.617393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:10.617393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:10.617393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:10.617393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:10.617393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:10.617393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:10.617393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:10.617393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:10.617393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:10.617393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:10.617393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:10.617393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:10.617393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:10.617393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:10.617393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:10.618360 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:10.618360 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:10.618360 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:10.618360 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:10.618360 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:10.618360 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:10.618360 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:10.618360 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:10.618360 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:10.618360 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:10.618360 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:06:10.618360 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:06:10.618360 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 14/28] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Aug 5 14:06:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:06:10.618360 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a444ca8-7b5d-47af-8af5-f9fe8b39d1bd None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:10.619225 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a444ca8-7b5d-47af-8af5-f9fe8b39d1bd None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:10.619225 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a444ca8-7b5d-47af-8af5-f9fe8b39d1bd None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:10.619225 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 15/29] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:10.619225 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a444ca8-7b5d-47af-8af5-f9fe8b39d1bd None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:10.730736 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a444ca8-7b5d-47af-8af5-f9fe8b39d1bd None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:10.826719 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 15/30] 10.222.0.26 () {60 vars in 1131 bytes} [Fri Aug 5 14:06:10 2022] POST /identity/v3/roles => generated 225 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:06:10.828011 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df70484-8498-4120-8e1a-12d741a3384b None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:10.850301 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:10.907495 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18b3ed5-2b6c-4b0e-b92f-44e78778a768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:10.908166 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b3ed5-2b6c-4b0e-b92f-44e78778a768 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:10.908454 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b3ed5-2b6c-4b0e-b92f-44e78778a768 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:10.908731 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b3ed5-2b6c-4b0e-b92f-44e78778a768 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:10.932941 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18b3ed5-2b6c-4b0e-b92f-44e78778a768 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:10.934429 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18b3ed5-2b6c-4b0e-b92f-44e78778a768 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:10.965601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-e18b3ed5-2b6c-4b0e-b92f-44e78778a768 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:06:10.965601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:10.965601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:10.965601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:10.965601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:10.965601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:10.965601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:10.965601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:10.965601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:10.965601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:10.965601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:10.965601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:10.965601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:06:10.965601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:06:10.965601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:06:10.967596 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:06:10.967596 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:10.967596 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:10.967596 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:10.967596 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:10.967596 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:10.967596 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:10.967596 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:10.967596 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:10.967596 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:10.967596 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:10.967596 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:10.967596 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:10.967596 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:10.967596 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:10.967596 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:10.967596 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:10.969635 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:06:10.969635 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:06:10.969635 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:06:10.969635 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:06:10.969635 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 16/31] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:06:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:06:10.973236 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:10.976312 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fa57d-66d5-4f27-b9e5-a74e448f90b6 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:10.976644 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fa57d-66d5-4f27-b9e5-a74e448f90b6 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:10.976916 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fa57d-66d5-4f27-b9e5-a74e448f90b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:11.013865 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-af0fa57d-66d5-4f27-b9e5-a74e448f90b6 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:11.015009 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-af0fa57d-66d5-4f27-b9e5-a74e448f90b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:11.051926 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df70484-8498-4120-8e1a-12d741a3384b None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:11.093586 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f925a195-4480-4753-a7d5-b8077d6c8273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:11.094227 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925a195-4480-4753-a7d5-b8077d6c8273 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:11.094510 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925a195-4480-4753-a7d5-b8077d6c8273 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:11.094785 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925a195-4480-4753-a7d5-b8077d6c8273 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:11.095622 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f925a195-4480-4753-a7d5-b8077d6c8273 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:11.095931 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f925a195-4480-4753-a7d5-b8077d6c8273 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:11.097603 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f925a195-4480-4753-a7d5-b8077d6c8273 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:11.109991 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 17/32] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:11.109991 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0fa57d-66d5-4f27-b9e5-a74e448f90b6 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:11.115215 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 16/33] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:06:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:11.121698 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f925a195-4480-4753-a7d5-b8077d6c8273 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:11.231137 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:11.243485 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0fa57d-66d5-4f27-b9e5-a74e448f90b6 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:11.246763 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f925a195-4480-4753-a7d5-b8077d6c8273 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:11.273465 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:11.279579 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592302ab-5ac6-4135-9049-0f33ce9a7a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:11.280393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592302ab-5ac6-4135-9049-0f33ce9a7a03 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:11.280711 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592302ab-5ac6-4135-9049-0f33ce9a7a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:11.280984 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592302ab-5ac6-4135-9049-0f33ce9a7a03 None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:11.281781 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592302ab-5ac6-4135-9049-0f33ce9a7a03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:11.282092 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592302ab-5ac6-4135-9049-0f33ce9a7a03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:11.282299 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba41d489-0bf8-4ba3-bf75-bf0ed3501cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:11.283001 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41d489-0bf8-4ba3-bf75-bf0ed3501cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:11.283283 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41d489-0bf8-4ba3-bf75-bf0ed3501cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:11.283544 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592302ab-5ac6-4135-9049-0f33ce9a7a03 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:11.283645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41d489-0bf8-4ba3-bf75-bf0ed3501cd3 None admin] PATH_INFO: `/v3/users/5e3fbbd9c18b4be39ebc01f40d1df40f` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:11.284887 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba41d489-0bf8-4ba3-bf75-bf0ed3501cd3 None admin] RBAC: Authorizing `identity:get_user(user_id=5e3fbbd9c18b4be39ebc01f40d1df40f)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:11.286321 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba41d489-0bf8-4ba3-bf75-bf0ed3501cd3 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:11.288653 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 17/34] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:06:11 2022] GET /identity/v3/users/5e3fbbd9c18b4be39ebc01f40d1df40f => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:06:11.289500 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba41d489-0bf8-4ba3-bf75-bf0ed3501cd3 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:11.300648 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 18/35] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:11.302088 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a49279-1b3f-4e52-8da5-ec2443515896 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:11.302449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a49279-1b3f-4e52-8da5-ec2443515896 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:11.302720 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a49279-1b3f-4e52-8da5-ec2443515896 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:11.305788 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 19/36] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:11.305788 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a49279-1b3f-4e52-8da5-ec2443515896 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:11.404135 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba41d489-0bf8-4ba3-bf75-bf0ed3501cd3 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:11.411501 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a49279-1b3f-4e52-8da5-ec2443515896 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:11.438811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcfcc3c-2582-48ec-80b4-64ae9f5716d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:11.439434 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcfcc3c-2582-48ec-80b4-64ae9f5716d6 None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:11.439712 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcfcc3c-2582-48ec-80b4-64ae9f5716d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:11.439982 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcfcc3c-2582-48ec-80b4-64ae9f5716d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:11.440723 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcfcc3c-2582-48ec-80b4-64ae9f5716d6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:11.442113 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcfcc3c-2582-48ec-80b4-64ae9f5716d6 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:11.447475 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f90375-072b-47ad-9b71-f832116a77fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:11.448120 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f90375-072b-47ad-9b71-f832116a77fd None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:11.448401 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f90375-072b-47ad-9b71-f832116a77fd None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:11.448705 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f90375-072b-47ad-9b71-f832116a77fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:11.450102 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f90375-072b-47ad-9b71-f832116a77fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:11.451534 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f90375-072b-47ad-9b71-f832116a77fd None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:11.453166 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 20/37] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:06:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:11.453903 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f90375-072b-47ad-9b71-f832116a77fd None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:11.472850 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:11.524532 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 18/38] 10.222.0.26 () {60 vars in 1131 bytes} [Fri Aug 5 14:06:11 2022] POST /identity/v3/roles => generated 219 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:06:11.524532 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5533678-8b4c-459c-841c-0ffef049ccc9 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:11.524532 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5533678-8b4c-459c-841c-0ffef049ccc9 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:11.524532 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5533678-8b4c-459c-841c-0ffef049ccc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:11.542001 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-e5533678-8b4c-459c-841c-0ffef049ccc9 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:11.543613 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e5533678-8b4c-459c-841c-0ffef049ccc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:11.574010 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f90375-072b-47ad-9b71-f832116a77fd None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:11.599874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:11.614078 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a5c316-5dea-41f3-9a13-522e95d34484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:11.614820 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5c316-5dea-41f3-9a13-522e95d34484 None admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:11.615108 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5c316-5dea-41f3-9a13-522e95d34484 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:11.615381 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5c316-5dea-41f3-9a13-522e95d34484 None admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:11.616243 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a5c316-5dea-41f3-9a13-522e95d34484 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:11.617900 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a5c316-5dea-41f3-9a13-522e95d34484 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:11.620132 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 19/39] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:11.621160 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5533678-8b4c-459c-841c-0ffef049ccc9 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:11.737888 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5533678-8b4c-459c-841c-0ffef049ccc9 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:11.772207 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d505af2-e1f0-49c6-99d1-fdf9039123bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:11.772924 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d505af2-e1f0-49c6-99d1-fdf9039123bf None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:11.773203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d505af2-e1f0-49c6-99d1-fdf9039123bf None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:11.773483 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d505af2-e1f0-49c6-99d1-fdf9039123bf None admin] PATH_INFO: `/v3/domains/default/users/5e3fbbd9c18b4be39ebc01f40d1df40f/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:11.775641 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d505af2-e1f0-49c6-99d1-fdf9039123bf None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=5e3fbbd9c18b4be39ebc01f40d1df40f, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:11.777180 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d505af2-e1f0-49c6-99d1-fdf9039123bf None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:11.849229 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 21/40] 10.222.0.26 () {60 vars in 1141 bytes} [Fri Aug 5 14:06:11 2022] POST /identity/v3/projects => generated 301 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:06:11.851350 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a25a7-ef13-4d02-b008-aec50c11924a None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:11.851642 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a25a7-ef13-4d02-b008-aec50c11924a None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:11.851917 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a25a7-ef13-4d02-b008-aec50c11924a None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:11.853207 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 22/41] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:11.853913 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a25a7-ef13-4d02-b008-aec50c11924a None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:11.871963 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 20/42] 10.222.0.26 () {58 vars in 1361 bytes} [Fri Aug 5 14:06:11 2022] PUT /identity/v3/domains/default/users/5e3fbbd9c18b4be39ebc01f40d1df40f/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:06:11.873713 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8aaeed-6a12-463d-815e-f93f3fe0b508 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:11.874441 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8aaeed-6a12-463d-815e-f93f3fe0b508 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:11.876671 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8aaeed-6a12-463d-815e-f93f3fe0b508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:11.901545 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-7b8aaeed-6a12-463d-815e-f93f3fe0b508 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:11.903615 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-7b8aaeed-6a12-463d-815e-f93f3fe0b508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:11.948747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:11.964863 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:12.002155 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a25a7-ef13-4d02-b008-aec50c11924a None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:12.004289 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 21/43] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:12.028315 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8aaeed-6a12-463d-815e-f93f3fe0b508 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:12.047785 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5874c3fc-8bf8-4f58-961d-8b23a0afd545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:12.051338 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874c3fc-8bf8-4f58-961d-8b23a0afd545 None admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:12.051338 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874c3fc-8bf8-4f58-961d-8b23a0afd545 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:12.051338 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874c3fc-8bf8-4f58-961d-8b23a0afd545 None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:12.051338 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5874c3fc-8bf8-4f58-961d-8b23a0afd545 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:12.052811 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5874c3fc-8bf8-4f58-961d-8b23a0afd545 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:12.129245 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 23/44] 10.222.0.26 () {60 vars in 1131 bytes} [Fri Aug 5 14:06:11 2022] POST /identity/v3/roles => generated 223 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:06:12.138918 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8aaeed-6a12-463d-815e-f93f3fe0b508 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:12.178266 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a64292-cba2-4f92-9efe-1af4a8f80e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:12.178981 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a64292-cba2-4f92-9efe-1af4a8f80e17 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:12.179286 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a64292-cba2-4f92-9efe-1af4a8f80e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:12.179631 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a64292-cba2-4f92-9efe-1af4a8f80e17 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:12.180925 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a64292-cba2-4f92-9efe-1af4a8f80e17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:12.182421 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a64292-cba2-4f92-9efe-1af4a8f80e17 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:12.184123 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 22/45] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:06:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:06:12.190239 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5874c3fc-8bf8-4f58-961d-8b23a0afd545 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:12.304276 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5874c3fc-8bf8-4f58-961d-8b23a0afd545 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:12.334019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:12.349240 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f15726e-e6aa-4cf9-9bb1-2250e6e9a2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:12.349868 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f15726e-e6aa-4cf9-9bb1-2250e6e9a2a4 None admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:12.350147 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f15726e-e6aa-4cf9-9bb1-2250e6e9a2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:12.350439 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f15726e-e6aa-4cf9-9bb1-2250e6e9a2a4 None admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:12.351365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f15726e-e6aa-4cf9-9bb1-2250e6e9a2a4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:12.352677 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f15726e-e6aa-4cf9-9bb1-2250e6e9a2a4 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:12.432699 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 24/46] 10.222.0.26 () {60 vars in 1141 bytes} [Fri Aug 5 14:06:12 2022] POST /identity/v3/projects => generated 312 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:06:13.925580 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ff1b9-db79-411a-aaff-6a80ff7a162b None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:13.926092 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ff1b9-db79-411a-aaff-6a80ff7a162b None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:13.926199 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ff1b9-db79-411a-aaff-6a80ff7a162b None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:13.927470 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 23/47] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:13.932674 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf4772-5653-4b97-a942-17ed23bac149 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:13.933091 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf4772-5653-4b97-a942-17ed23bac149 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:13.933262 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf4772-5653-4b97-a942-17ed23bac149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:13.951765 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-dfbf4772-5653-4b97-a942-17ed23bac149 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:13.953285 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-dfbf4772-5653-4b97-a942-17ed23bac149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:14.017679 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 25/48] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:14.040537 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ff1b9-db79-411a-aaff-6a80ff7a162b None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:14.084045 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:14.140025 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ff1b9-db79-411a-aaff-6a80ff7a162b None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:14.172525 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7775d26-63fe-4cd5-ad65-c9a91e9c9079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:14.173158 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7775d26-63fe-4cd5-ad65-c9a91e9c9079 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:14.173436 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7775d26-63fe-4cd5-ad65-c9a91e9c9079 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:14.173709 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7775d26-63fe-4cd5-ad65-c9a91e9c9079 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:14.184019 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7775d26-63fe-4cd5-ad65-c9a91e9c9079 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:14.188893 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-f7775d26-63fe-4cd5-ad65-c9a91e9c9079 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:06:14.188893 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:14.188893 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:14.188893 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:14.188893 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:14.188893 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:14.188893 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:14.188893 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:14.188893 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:14.188893 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:14.188893 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:14.188893 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:14.188893 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:14.188893 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:14.188893 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:14.188893 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:14.189874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:14.189874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:14.189874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:14.189874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:14.189874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:14.189874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:14.189874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:14.189874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:14.189874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:14.189874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:14.189874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:14.189874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:14.189874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:14.189874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:14.189874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:14.189874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:14.190677 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:14.190677 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:14.190677 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:14.190677 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:14.190677 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:14.190677 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:14.190677 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:14.190677 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:14.190677 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:14.190677 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:14.190677 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:06:14.190677 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:06:14.190677 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 24/49] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Aug 5 14:06:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:06:14.196046 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf4772-5653-4b97-a942-17ed23bac149 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:14.293361 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf4772-5653-4b97-a942-17ed23bac149 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:14.303979 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:14.326216 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533b1c0-cf89-4d6d-bde6-13cc6bbd70d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:14.326850 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533b1c0-cf89-4d6d-bde6-13cc6bbd70d9 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:14.327330 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533b1c0-cf89-4d6d-bde6-13cc6bbd70d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:14.327772 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533b1c0-cf89-4d6d-bde6-13cc6bbd70d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:14.328880 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533b1c0-cf89-4d6d-bde6-13cc6bbd70d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:14.329239 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533b1c0-cf89-4d6d-bde6-13cc6bbd70d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:14.330612 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533b1c0-cf89-4d6d-bde6-13cc6bbd70d9 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:14.342991 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 26/50] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:06:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:14.348636 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7775d26-63fe-4cd5-ad65-c9a91e9c9079 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:14.439300 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b7bfe-e020-42b7-823d-2de345ee61cc None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:14.439502 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b7bfe-e020-42b7-823d-2de345ee61cc None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:14.439774 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b7bfe-e020-42b7-823d-2de345ee61cc None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:14.441149 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 27/51] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:06:14.446206 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f19994e-c584-4e94-8335-9ab7af3f3b96 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:14.446492 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f19994e-c584-4e94-8335-9ab7af3f3b96 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:14.446777 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f19994e-c584-4e94-8335-9ab7af3f3b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:14.450110 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7775d26-63fe-4cd5-ad65-c9a91e9c9079 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:14.451617 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:14.464599 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-3f19994e-c584-4e94-8335-9ab7af3f3b96 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:14.465820 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-3f19994e-c584-4e94-8335-9ab7af3f3b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:14.485667 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653144f-b967-4929-b98f-97901c37d12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:14.486301 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653144f-b967-4929-b98f-97901c37d12e None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:14.486578 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653144f-b967-4929-b98f-97901c37d12e None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:14.486846 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653144f-b967-4929-b98f-97901c37d12e None admin] PATH_INFO: `/v3/users/5e3fbbd9c18b4be39ebc01f40d1df40f` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:14.488000 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a653144f-b967-4929-b98f-97901c37d12e None admin] RBAC: Authorizing `identity:get_user(user_id=5e3fbbd9c18b4be39ebc01f40d1df40f)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:14.489472 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a653144f-b967-4929-b98f-97901c37d12e None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:14.491699 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 25/52] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:06:14 2022] GET /identity/v3/users/5e3fbbd9c18b4be39ebc01f40d1df40f => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:14.496998 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653144f-b967-4929-b98f-97901c37d12e None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:14.533356 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 28/53] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 14:06:14.556700 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f19994e-c584-4e94-8335-9ab7af3f3b96 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:14.597389 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653144f-b967-4929-b98f-97901c37d12e None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:14.631288 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbddf9df-fd56-40e7-8f11-b5bf51f5eba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:14.632640 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbddf9df-fd56-40e7-8f11-b5bf51f5eba8 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:14.632885 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbddf9df-fd56-40e7-8f11-b5bf51f5eba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:14.633155 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbddf9df-fd56-40e7-8f11-b5bf51f5eba8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:14.634336 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbddf9df-fd56-40e7-8f11-b5bf51f5eba8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:14.635789 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbddf9df-fd56-40e7-8f11-b5bf51f5eba8 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:14.637582 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 26/54] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:06:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:06:14.643004 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbddf9df-fd56-40e7-8f11-b5bf51f5eba8 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:14.657895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f19994e-c584-4e94-8335-9ab7af3f3b96 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:14.659414 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:14.683422 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:14.691397 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cddb606-8ed4-4dc5-9f16-de5be36d4888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:14.691979 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cddb606-8ed4-4dc5-9f16-de5be36d4888 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:14.692260 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cddb606-8ed4-4dc5-9f16-de5be36d4888 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:14.692568 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cddb606-8ed4-4dc5-9f16-de5be36d4888 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:14.693767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cddb606-8ed4-4dc5-9f16-de5be36d4888 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:14.695836 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cddb606-8ed4-4dc5-9f16-de5be36d4888 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:14.697499 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 29/55] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:06:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:06:14.703806 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cddb606-8ed4-4dc5-9f16-de5be36d4888 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:14.742740 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbddf9df-fd56-40e7-8f11-b5bf51f5eba8 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:14.777195 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26d59e-2153-4708-8b56-c7f3151cad77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:14.777852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26d59e-2153-4708-8b56-c7f3151cad77 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:14.778134 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26d59e-2153-4708-8b56-c7f3151cad77 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:14.778410 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26d59e-2153-4708-8b56-c7f3151cad77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:14.779206 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef26d59e-2153-4708-8b56-c7f3151cad77 None admin] RBAC: Adding query filter params (role.id=299c794c7acb4a5ca36288c3191aad10, scope.domain.id=default, user.id=5e3fbbd9c18b4be39ebc01f40d1df40f) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:14.779517 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef26d59e-2153-4708-8b56-c7f3151cad77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:14.780999 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef26d59e-2153-4708-8b56-c7f3151cad77 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:14.786852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 27/56] 10.222.0.26 () {56 vars in 1322 bytes} [Fri Aug 5 14:06:14 2022] GET /identity/v3/role_assignments?user.id=5e3fbbd9c18b4be39ebc01f40d1df40f&scope.domain.id=default&role.id=299c794c7acb4a5ca36288c3191aad10 => generated 530 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:06:14.802110 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cddb606-8ed4-4dc5-9f16-de5be36d4888 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:14.841554 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7fc858-a33e-4835-82b5-43d7af73efa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:14.842208 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fc858-a33e-4835-82b5-43d7af73efa8 None admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:14.842495 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fc858-a33e-4835-82b5-43d7af73efa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:14.842777 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fc858-a33e-4835-82b5-43d7af73efa8 None admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:14.843488 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7fc858-a33e-4835-82b5-43d7af73efa8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:14.845051 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7fc858-a33e-4835-82b5-43d7af73efa8 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:14.935076 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 30/57] 10.222.0.26 () {60 vars in 1140 bytes} [Fri Aug 5 14:06:14 2022] POST /identity/v3/projects => generated 298 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:06:16.942990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db12c9-2912-4a5b-a985-daecb63cb0eb None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:16.943544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db12c9-2912-4a5b-a985-daecb63cb0eb None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:16.943544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db12c9-2912-4a5b-a985-daecb63cb0eb None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:16.945508 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 28/58] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:16.950866 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de8efef-7950-4943-a862-5588bab1ac68 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:16.951297 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de8efef-7950-4943-a862-5588bab1ac68 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:16.951405 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de8efef-7950-4943-a862-5588bab1ac68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:16.971540 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-3de8efef-7950-4943-a862-5588bab1ac68 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:16.972795 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-3de8efef-7950-4943-a862-5588bab1ac68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:16.991109 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:17.044242 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 31/59] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:17.068529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0db12c9-2912-4a5b-a985-daecb63cb0eb None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:17.210813 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0db12c9-2912-4a5b-a985-daecb63cb0eb None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:17.213137 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:17.257883 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e352551-c1c3-47bf-b9a5-9e2ab03ec9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:17.258520 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e352551-c1c3-47bf-b9a5-9e2ab03ec9e2 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:17.258799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e352551-c1c3-47bf-b9a5-9e2ab03ec9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:17.259070 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e352551-c1c3-47bf-b9a5-9e2ab03ec9e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:17.261517 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e352551-c1c3-47bf-b9a5-9e2ab03ec9e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:17.263372 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e352551-c1c3-47bf-b9a5-9e2ab03ec9e2 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:17.265104 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 29/60] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:06:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:17.270951 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de8efef-7950-4943-a862-5588bab1ac68 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:17.375169 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de8efef-7950-4943-a862-5588bab1ac68 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:17.409776 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9cd730-78ea-41a5-8c9a-1968d838c1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:17.410394 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9cd730-78ea-41a5-8c9a-1968d838c1ea None admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:17.410673 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9cd730-78ea-41a5-8c9a-1968d838c1ea None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:17.410947 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9cd730-78ea-41a5-8c9a-1968d838c1ea None admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:17.411638 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9cd730-78ea-41a5-8c9a-1968d838c1ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:17.413257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9cd730-78ea-41a5-8c9a-1968d838c1ea None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:17.511016 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 32/61] 10.222.0.26 () {60 vars in 1132 bytes} [Fri Aug 5 14:06:17 2022] POST /identity/v3/users => generated 275 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:06:20.395057 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4e66c-4840-4b25-ba77-3a2d642bc9bb None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:20.396055 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4e66c-4840-4b25-ba77-3a2d642bc9bb None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:20.396574 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4e66c-4840-4b25-ba77-3a2d642bc9bb None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:20.399026 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 30/62] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:20.404291 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f658a2d-4f36-48bf-9dc1-762b64750861 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:20.405199 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f658a2d-4f36-48bf-9dc1-762b64750861 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:20.405680 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f658a2d-4f36-48bf-9dc1-762b64750861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:20.433644 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-3f658a2d-4f36-48bf-9dc1-762b64750861 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:20.435202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-3f658a2d-4f36-48bf-9dc1-762b64750861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:20.437457 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:20.535070 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 33/63] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:20.574195 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e4e66c-4840-4b25-ba77-3a2d642bc9bb None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:20.700306 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e4e66c-4840-4b25-ba77-3a2d642bc9bb None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:20.743133 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaff3f2-d443-49b3-b98a-7699e43b39bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:20.743784 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaff3f2-d443-49b3-b98a-7699e43b39bf None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:20.744065 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaff3f2-d443-49b3-b98a-7699e43b39bf None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:20.744337 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaff3f2-d443-49b3-b98a-7699e43b39bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:20.760019 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaff3f2-d443-49b3-b98a-7699e43b39bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:20.769055 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-dfaff3f2-d443-49b3-b98a-7699e43b39bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 14:06:20.769055 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:20.769055 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:20.769055 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:20.769055 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:20.769055 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:20.769055 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:20.769055 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:20.769055 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:20.769055 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:20.769055 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:20.769055 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:20.769055 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:20.769055 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:20.769055 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:20.769055 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:20.769942 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:20.769942 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:20.769942 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:20.769942 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:20.769942 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:20.769942 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:20.769942 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:20.769942 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:20.769942 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:20.769942 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:20.769942 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:20.769942 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:20.769942 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:20.769942 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:20.769942 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:20.769942 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:20.770719 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:20.770719 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:20.770719 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:20.770719 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:20.770719 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:20.770719 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:20.770719 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:20.770719 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:20.770719 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:20.770719 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:20.770719 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 14:06:20.770719 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:06:20.770719 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 31/64] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:06:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:06:20.778296 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f658a2d-4f36-48bf-9dc1-762b64750861 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:20.786192 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48549f7c-eec2-4167-a1c5-1b45056b3737 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:20.786479 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48549f7c-eec2-4167-a1c5-1b45056b3737 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:20.786757 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48549f7c-eec2-4167-a1c5-1b45056b3737 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:20.788793 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 32/65] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:06:20.796057 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcffe08-56fa-406d-a679-614f3c9375ca None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:20.796269 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcffe08-56fa-406d-a679-614f3c9375ca None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:20.796538 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcffe08-56fa-406d-a679-614f3c9375ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:20.819282 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-cdcffe08-56fa-406d-a679-614f3c9375ca None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:20.820791 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-cdcffe08-56fa-406d-a679-614f3c9375ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:20.848810 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:20.922658 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 33/66] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 14:06:20.936842 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f658a2d-4f36-48bf-9dc1-762b64750861 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:20.966025 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcffe08-56fa-406d-a679-614f3c9375ca None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:20.980219 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667023e3-4e32-47f2-bd8e-47c9551f081e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:20.980872 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667023e3-4e32-47f2-bd8e-47c9551f081e None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:20.981153 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667023e3-4e32-47f2-bd8e-47c9551f081e None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:20.981425 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667023e3-4e32-47f2-bd8e-47c9551f081e None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:20.982144 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667023e3-4e32-47f2-bd8e-47c9551f081e None admin] RBAC: Adding query filter params (name=member) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:20.982445 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667023e3-4e32-47f2-bd8e-47c9551f081e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:20.983809 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667023e3-4e32-47f2-bd8e-47c9551f081e None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:20.992977 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:20.996724 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 34/67] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Aug 5 14:06:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:20.998291 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667023e3-4e32-47f2-bd8e-47c9551f081e None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:21.092286 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcffe08-56fa-406d-a679-614f3c9375ca None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:21.129991 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f48387-e4d9-4634-be2c-084c24f11510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:21.130647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f48387-e4d9-4634-be2c-084c24f11510 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:21.130928 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f48387-e4d9-4634-be2c-084c24f11510 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:21.131204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f48387-e4d9-4634-be2c-084c24f11510 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:21.143654 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f48387-e4d9-4634-be2c-084c24f11510 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:21.144774 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667023e3-4e32-47f2-bd8e-47c9551f081e None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:21.149357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-56f48387-e4d9-4634-be2c-084c24f11510 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 14:06:21.149357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:21.149357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:21.149357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:21.149357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:21.149357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:21.149357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:21.149357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:21.149357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:21.149357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:21.149357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:21.149357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:21.149357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:21.149357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:21.149357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:21.149357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:21.150204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:21.150204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:21.150204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:21.150204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:21.150204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:21.150204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:21.150204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:21.150204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:21.150204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:21.150204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:21.150204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:21.150204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:21.150204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:21.150204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:21.150204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:21.150204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:21.151087 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:21.151087 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:21.151087 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:21.151087 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:21.151087 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:21.151087 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:21.151087 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:21.151087 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:21.151087 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:21.151087 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:21.151087 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 14:06:21.151087 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:06:21.151087 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 34/68] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Aug 5 14:06:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:06:21.151736 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8346120-42f0-482b-b7cf-d3064d38b4c1 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:21.152286 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8346120-42f0-482b-b7cf-d3064d38b4c1 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:21.152286 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8346120-42f0-482b-b7cf-d3064d38b4c1 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:21.162139 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 35/69] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:21.162139 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8346120-42f0-482b-b7cf-d3064d38b4c1 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:21.183676 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:21.186079 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24b6ac3-c46b-4d7d-b90d-35b92225182f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:21.186735 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b6ac3-c46b-4d7d-b90d-35b92225182f None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:21.187022 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b6ac3-c46b-4d7d-b90d-35b92225182f None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:21.187303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b6ac3-c46b-4d7d-b90d-35b92225182f None admin] PATH_INFO: `/v3/users/da4b0c24aa284fd59788ab74c424af63` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:21.206341 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24b6ac3-c46b-4d7d-b90d-35b92225182f None admin] RBAC: Authorizing `identity:get_user(user_id=da4b0c24aa284fd59788ab74c424af63)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:21.207813 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24b6ac3-c46b-4d7d-b90d-35b92225182f None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:21.209552 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 35/70] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:06:20 2022] GET /identity/v3/users/da4b0c24aa284fd59788ab74c424af63 => generated 275 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:06:21.211704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91118f-c56a-4d22-8507-0fb530eac696 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:21.211996 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91118f-c56a-4d22-8507-0fb530eac696 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:21.212275 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91118f-c56a-4d22-8507-0fb530eac696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:21.231071 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-9f91118f-c56a-4d22-8507-0fb530eac696 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:21.232202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-9f91118f-c56a-4d22-8507-0fb530eac696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:21.278898 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8346120-42f0-482b-b7cf-d3064d38b4c1 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:21.289236 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:21.302765 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 36/71] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:21.306464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456ba3e-7f5b-4d66-a1e8-26f6911d0b09 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:21.306977 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456ba3e-7f5b-4d66-a1e8-26f6911d0b09 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:21.307723 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456ba3e-7f5b-4d66-a1e8-26f6911d0b09 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:21.309880 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 37/72] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:21.310779 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456ba3e-7f5b-4d66-a1e8-26f6911d0b09 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:21.318832 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6440f7c2-fd08-4032-838e-1b0bc561b3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:21.319647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6440f7c2-fd08-4032-838e-1b0bc561b3e8 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:21.320024 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6440f7c2-fd08-4032-838e-1b0bc561b3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:21.320619 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6440f7c2-fd08-4032-838e-1b0bc561b3e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:21.321515 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6440f7c2-fd08-4032-838e-1b0bc561b3e8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:21.321883 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6440f7c2-fd08-4032-838e-1b0bc561b3e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:21.323411 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6440f7c2-fd08-4032-838e-1b0bc561b3e8 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:21.332093 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 36/73] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:06:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:21.333577 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60a8c1-87f7-4ef1-90dc-5adecae3d718 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:21.333970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60a8c1-87f7-4ef1-90dc-5adecae3d718 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:21.334342 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60a8c1-87f7-4ef1-90dc-5adecae3d718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:21.353063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-dd60a8c1-87f7-4ef1-90dc-5adecae3d718 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:21.354254 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-dd60a8c1-87f7-4ef1-90dc-5adecae3d718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:21.419320 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 37/74] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:21.420144 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd60a8c1-87f7-4ef1-90dc-5adecae3d718 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:21.421600 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456ba3e-7f5b-4d66-a1e8-26f6911d0b09 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:21.440741 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:21.458729 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019f7f76-81fc-462d-b84a-ae9ac3a443e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:21.466153 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f7f76-81fc-462d-b84a-ae9ac3a443e9 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:21.466153 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f7f76-81fc-462d-b84a-ae9ac3a443e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:21.466153 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f7f76-81fc-462d-b84a-ae9ac3a443e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:21.515029 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:21.528553 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019f7f76-81fc-462d-b84a-ae9ac3a443e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:21.529339 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd60a8c1-87f7-4ef1-90dc-5adecae3d718 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:21.530086 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019f7f76-81fc-462d-b84a-ae9ac3a443e9 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:21.562367 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4888fe70-341d-4476-8968-8cc9c32ae61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:21.563015 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888fe70-341d-4476-8968-8cc9c32ae61b None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:21.563300 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888fe70-341d-4476-8968-8cc9c32ae61b None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:21.563577 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888fe70-341d-4476-8968-8cc9c32ae61b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:21.574185 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4888fe70-341d-4476-8968-8cc9c32ae61b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:21.579529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-4888fe70-341d-4476-8968-8cc9c32ae61b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:06:21.579529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:21.579529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:21.579529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:21.579529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:21.579529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:21.579529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:21.579529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:21.579529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:21.579529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:21.579529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:21.579529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:21.579529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:21.579529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:21.579529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:21.579529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:21.580376 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:21.580376 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:21.580376 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:21.580376 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:21.580376 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:21.580376 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:21.580376 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:21.580376 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:21.580376 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:21.580376 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:21.580376 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:21.580376 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:21.580376 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:21.580376 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:21.580376 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:21.580376 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:21.581465 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:21.581465 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:21.581465 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:21.581465 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:21.581465 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:21.581465 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:21.581465 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:21.581465 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:21.581465 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:21.581465 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:21.581465 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:06:21.581465 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:06:21.581465 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 38/75] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Aug 5 14:06:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:06:21.581465 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4888fe70-341d-4476-8968-8cc9c32ae61b None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:21.634157 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 38/76] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:06:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 14:06:21.635535 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4390f-a2bf-4fac-9c75-22035ba0cb71 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:21.635823 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4390f-a2bf-4fac-9c75-22035ba0cb71 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:21.636091 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4390f-a2bf-4fac-9c75-22035ba0cb71 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:21.637355 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 39/77] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:21.638042 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe4390f-a2bf-4fac-9c75-22035ba0cb71 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:21.652458 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:21.680896 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4888fe70-341d-4476-8968-8cc9c32ae61b None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:21.714866 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb5aea6-116f-4d55-8222-aec38edd6a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:21.715501 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5aea6-116f-4d55-8222-aec38edd6a24 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:21.715780 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5aea6-116f-4d55-8222-aec38edd6a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:21.716058 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5aea6-116f-4d55-8222-aec38edd6a24 None admin] PATH_INFO: `/v3/users/da4b0c24aa284fd59788ab74c424af63` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:21.717270 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb5aea6-116f-4d55-8222-aec38edd6a24 None admin] RBAC: Authorizing `identity:get_user(user_id=da4b0c24aa284fd59788ab74c424af63)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:21.718704 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb5aea6-116f-4d55-8222-aec38edd6a24 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:21.720287 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 39/78] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:06:21 2022] GET /identity/v3/users/da4b0c24aa284fd59788ab74c424af63 => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:21.725267 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb5aea6-116f-4d55-8222-aec38edd6a24 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:21.725921 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:21.739683 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe4390f-a2bf-4fac-9c75-22035ba0cb71 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:21.775692 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816617a-791e-4e14-beba-6ab337db5efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:21.776257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816617a-791e-4e14-beba-6ab337db5efe None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:21.776564 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816617a-791e-4e14-beba-6ab337db5efe None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:21.776836 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816617a-791e-4e14-beba-6ab337db5efe None admin] PATH_INFO: `/v3/roles/member` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:21.786929 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0816617a-791e-4e14-beba-6ab337db5efe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:21.792151 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-0816617a-791e-4e14-beba-6ab337db5efe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 14:06:21.792151 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:21.792151 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:21.792151 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:21.792151 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:21.792151 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:21.792151 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:21.792151 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:21.792151 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:21.792151 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:21.792151 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:21.792151 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:21.792151 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:21.792151 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:21.792151 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:21.792151 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:21.793061 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:21.793061 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:21.793061 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:21.793061 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:21.793061 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:21.793061 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:21.793061 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:21.793061 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:21.793061 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:21.793061 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:21.793061 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:21.793061 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:21.793061 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:21.793061 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:21.793061 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:21.793061 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:21.793842 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:21.793842 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:21.793842 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:21.793842 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:21.793842 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:21.793842 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:21.793842 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:21.793842 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:21.793842 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:21.793842 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:21.793842 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 14:06:21.793842 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:06:21.793842 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 40/79] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:06:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:06:21.794480 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816617a-791e-4e14-beba-6ab337db5efe None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:21.827447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb5aea6-116f-4d55-8222-aec38edd6a24 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:21.859937 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4f74c0-a355-40fb-8294-eda00b71ae13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:21.860585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f74c0-a355-40fb-8294-eda00b71ae13 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:21.860874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f74c0-a355-40fb-8294-eda00b71ae13 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:21.861151 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f74c0-a355-40fb-8294-eda00b71ae13 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:21.861866 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4f74c0-a355-40fb-8294-eda00b71ae13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:21.862172 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4f74c0-a355-40fb-8294-eda00b71ae13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:21.863569 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4f74c0-a355-40fb-8294-eda00b71ae13 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:21.872519 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 40/80] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:06:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:21.874191 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8916b3-f1a4-4290-bf8d-fab044a1b3a9 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:21.874653 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8916b3-f1a4-4290-bf8d-fab044a1b3a9 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:21.875091 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8916b3-f1a4-4290-bf8d-fab044a1b3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:21.881522 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:21.893483 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-9f8916b3-f1a4-4290-bf8d-fab044a1b3a9 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:21.894673 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-9f8916b3-f1a4-4290-bf8d-fab044a1b3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:21.894919 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816617a-791e-4e14-beba-6ab337db5efe None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:21.929195 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7917b-6585-4201-91da-87682e4843f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:21.930075 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7917b-6585-4201-91da-87682e4843f3 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:21.930356 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7917b-6585-4201-91da-87682e4843f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:21.930630 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7917b-6585-4201-91da-87682e4843f3 None admin] PATH_INFO: `/v3/projects/580a155c67b742b9bc398f50556f26cf` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:21.931799 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f7917b-6585-4201-91da-87682e4843f3 None admin] RBAC: Authorizing `identity:get_project(project_id=580a155c67b742b9bc398f50556f26cf)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:21.933264 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f7917b-6585-4201-91da-87682e4843f3 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:21.934924 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 41/81] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:06:21 2022] GET /identity/v3/projects/580a155c67b742b9bc398f50556f26cf => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:21.935678 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7917b-6585-4201-91da-87682e4843f3 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:21.953691 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:21.961428 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 41/82] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:21.965725 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8916b3-f1a4-4290-bf8d-fab044a1b3a9 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:22.034517 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7917b-6585-4201-91da-87682e4843f3 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:22.068118 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8916b3-f1a4-4290-bf8d-fab044a1b3a9 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:22.068479 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f13f9fb-bb29-4592-8f5b-72d0c886cf14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:22.069099 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13f9fb-bb29-4592-8f5b-72d0c886cf14 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:22.069569 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13f9fb-bb29-4592-8f5b-72d0c886cf14 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:22.070029 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13f9fb-bb29-4592-8f5b-72d0c886cf14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:22.097477 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:22.102965 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56478d-d7d6-4279-afd6-efa44f4e642e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:22.103575 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56478d-d7d6-4279-afd6-efa44f4e642e None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:22.103851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56478d-d7d6-4279-afd6-efa44f4e642e None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:22.104129 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56478d-d7d6-4279-afd6-efa44f4e642e None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:22.104876 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea56478d-d7d6-4279-afd6-efa44f4e642e None admin] RBAC: Adding query filter params (name=member) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:22.105181 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea56478d-d7d6-4279-afd6-efa44f4e642e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:22.106547 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea56478d-d7d6-4279-afd6-efa44f4e642e None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:22.114429 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 42/83] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Aug 5 14:06:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:22.115316 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56478d-d7d6-4279-afd6-efa44f4e642e None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:22.138159 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f13f9fb-bb29-4592-8f5b-72d0c886cf14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:22.139621 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f13f9fb-bb29-4592-8f5b-72d0c886cf14 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:22.157435 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:22.218608 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56478d-d7d6-4279-afd6-efa44f4e642e None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:22.238426 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 42/84] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:06:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 14:06:22.239271 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f13f9fb-bb29-4592-8f5b-72d0c886cf14 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:22.258644 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1e0aa8-2ce4-43db-8132-d77e7039dd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:22.259287 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e0aa8-2ce4-43db-8132-d77e7039dd55 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:22.259569 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e0aa8-2ce4-43db-8132-d77e7039dd55 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:22.259843 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e0aa8-2ce4-43db-8132-d77e7039dd55 None admin] PATH_INFO: `/v3/users/5e3fbbd9c18b4be39ebc01f40d1df40f` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:22.261042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1e0aa8-2ce4-43db-8132-d77e7039dd55 None admin] RBAC: Authorizing `identity:get_user(user_id=5e3fbbd9c18b4be39ebc01f40d1df40f)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:22.262472 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1e0aa8-2ce4-43db-8132-d77e7039dd55 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:22.264059 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 43/85] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:06:22 2022] GET /identity/v3/users/5e3fbbd9c18b4be39ebc01f40d1df40f => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:22.264850 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1e0aa8-2ce4-43db-8132-d77e7039dd55 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:22.324358 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:22.337151 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f13f9fb-bb29-4592-8f5b-72d0c886cf14 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:22.353655 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:22.368985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1e0aa8-2ce4-43db-8132-d77e7039dd55 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:22.371812 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329278d2-1eea-426b-9a80-16268d917013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:22.372549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329278d2-1eea-426b-9a80-16268d917013 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:22.372839 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329278d2-1eea-426b-9a80-16268d917013 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:22.373113 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329278d2-1eea-426b-9a80-16268d917013 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:22.374158 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329278d2-1eea-426b-9a80-16268d917013 None admin] RBAC: Adding query filter params (role.id=29967df108484a30a4c006cde2ae2940, scope.project.id=580a155c67b742b9bc398f50556f26cf, user.id=da4b0c24aa284fd59788ab74c424af63) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:22.374469 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329278d2-1eea-426b-9a80-16268d917013 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:22.375900 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329278d2-1eea-426b-9a80-16268d917013 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:22.384326 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 43/86] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:06:22 2022] GET /identity/v3/role_assignments?user.id=da4b0c24aa284fd59788ab74c424af63&scope.project.id=580a155c67b742b9bc398f50556f26cf&role.id=29967df108484a30a4c006cde2ae2940 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:22.385163 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329278d2-1eea-426b-9a80-16268d917013 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:22.410789 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede8860e-9a35-440b-90e0-2c1b7532fb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:22.411461 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede8860e-9a35-440b-90e0-2c1b7532fb42 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:22.411753 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede8860e-9a35-440b-90e0-2c1b7532fb42 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:22.412625 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede8860e-9a35-440b-90e0-2c1b7532fb42 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:22.414614 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede8860e-9a35-440b-90e0-2c1b7532fb42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:22.416089 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede8860e-9a35-440b-90e0-2c1b7532fb42 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:22.417859 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 44/87] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:06:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:22.423750 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede8860e-9a35-440b-90e0-2c1b7532fb42 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:22.503093 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329278d2-1eea-426b-9a80-16268d917013 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:22.537290 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede8860e-9a35-440b-90e0-2c1b7532fb42 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:22.549500 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435cdc73-1f11-4da5-a46a-c897fbf00bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:22.550151 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cdc73-1f11-4da5-a46a-c897fbf00bca None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:22.550435 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cdc73-1f11-4da5-a46a-c897fbf00bca None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:22.550711 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cdc73-1f11-4da5-a46a-c897fbf00bca None admin] PATH_INFO: `/v3/users/da4b0c24aa284fd59788ab74c424af63` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:22.551885 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435cdc73-1f11-4da5-a46a-c897fbf00bca None admin] RBAC: Authorizing `identity:get_user(user_id=da4b0c24aa284fd59788ab74c424af63)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:22.553426 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435cdc73-1f11-4da5-a46a-c897fbf00bca None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:22.555499 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 44/88] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:06:22 2022] GET /identity/v3/users/da4b0c24aa284fd59788ab74c424af63 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:22.556277 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435cdc73-1f11-4da5-a46a-c897fbf00bca None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:22.578659 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:22.585664 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb7c89-e647-4660-ba94-f152785ad622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:22.591506 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb7c89-e647-4660-ba94-f152785ad622 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:22.591506 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb7c89-e647-4660-ba94-f152785ad622 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:22.591506 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb7c89-e647-4660-ba94-f152785ad622 None admin] PATH_INFO: `/v3/projects/580a155c67b742b9bc398f50556f26cf` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:22.591506 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb7c89-e647-4660-ba94-f152785ad622 None admin] RBAC: Authorizing `identity:get_project(project_id=580a155c67b742b9bc398f50556f26cf)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:22.591506 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb7c89-e647-4660-ba94-f152785ad622 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:22.591506 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 45/89] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:06:22 2022] GET /identity/v3/projects/580a155c67b742b9bc398f50556f26cf => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:22.592229 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb7c89-e647-4660-ba94-f152785ad622 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:22.612898 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:22.667615 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435cdc73-1f11-4da5-a46a-c897fbf00bca None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:22.702532 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c98a9-0c77-4de3-942b-a6be77146aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:22.703399 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c98a9-0c77-4de3-942b-a6be77146aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:22.703683 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c98a9-0c77-4de3-942b-a6be77146aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:22.703958 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c98a9-0c77-4de3-942b-a6be77146aa3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:22.708380 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb7c89-e647-4660-ba94-f152785ad622 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:22.746176 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011b2cc-3763-4c99-8970-236ba1285f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:22.746796 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011b2cc-3763-4c99-8970-236ba1285f2f None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:22.747077 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011b2cc-3763-4c99-8970-236ba1285f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:22.747350 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011b2cc-3763-4c99-8970-236ba1285f2f None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:22.748034 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c011b2cc-3763-4c99-8970-236ba1285f2f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:22.749776 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c011b2cc-3763-4c99-8970-236ba1285f2f None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:22.777799 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069c98a9-0c77-4de3-942b-a6be77146aa3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:22.780577 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069c98a9-0c77-4de3-942b-a6be77146aa3 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:22.826022 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:22.834673 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 46/90] 10.222.0.26 () {60 vars in 1132 bytes} [Fri Aug 5 14:06:22 2022] POST /identity/v3/users => generated 289 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:06:22.836966 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011b2cc-3763-4c99-8970-236ba1285f2f None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:22.893343 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 45/91] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:06:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 14:06:22.894216 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c98a9-0c77-4de3-942b-a6be77146aa3 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:22.921457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:22.990800 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011b2cc-3763-4c99-8970-236ba1285f2f None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:23.016862 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c98a9-0c77-4de3-942b-a6be77146aa3 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:23.042578 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f0b663-fad6-4e0b-a744-5a65fef210e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:23.043740 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0b663-fad6-4e0b-a744-5a65fef210e7 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:23.044030 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0b663-fad6-4e0b-a744-5a65fef210e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:23.044347 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0b663-fad6-4e0b-a744-5a65fef210e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:23.059236 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22751096-bc5c-4241-97eb-43880fa0baca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:23.060219 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22751096-bc5c-4241-97eb-43880fa0baca None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:23.061042 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22751096-bc5c-4241-97eb-43880fa0baca None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:23.061776 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22751096-bc5c-4241-97eb-43880fa0baca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:23.062585 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22751096-bc5c-4241-97eb-43880fa0baca None admin] RBAC: Adding query filter params (role.id=199fd3093a824ba585b1135c8bf363be, scope.project.id=580a155c67b742b9bc398f50556f26cf, user.id=da4b0c24aa284fd59788ab74c424af63) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:23.062899 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22751096-bc5c-4241-97eb-43880fa0baca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:23.065315 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22751096-bc5c-4241-97eb-43880fa0baca None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:23.072674 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 46/92] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:06:22 2022] GET /identity/v3/role_assignments?user.id=da4b0c24aa284fd59788ab74c424af63&scope.project.id=580a155c67b742b9bc398f50556f26cf&role.id=199fd3093a824ba585b1135c8bf363be => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:23.074719 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22751096-bc5c-4241-97eb-43880fa0baca None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:23.095007 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:23.137112 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f0b663-fad6-4e0b-a744-5a65fef210e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:23.139313 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f0b663-fad6-4e0b-a744-5a65fef210e7 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:23.214413 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:23.221319 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22751096-bc5c-4241-97eb-43880fa0baca None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:23.269721 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e1e2e6-f21f-4538-8186-450055c515b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:23.270300 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1e2e6-f21f-4538-8186-450055c515b6 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:23.270772 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1e2e6-f21f-4538-8186-450055c515b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:23.271060 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1e2e6-f21f-4538-8186-450055c515b6 None admin] PATH_INFO: `/v3/projects/580a155c67b742b9bc398f50556f26cf` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:23.272920 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e1e2e6-f21f-4538-8186-450055c515b6 None admin] RBAC: Authorizing `identity:get_project(project_id=580a155c67b742b9bc398f50556f26cf)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:23.274806 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e1e2e6-f21f-4538-8186-450055c515b6 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:23.277698 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 47/93] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:06:23 2022] GET /identity/v3/projects/580a155c67b742b9bc398f50556f26cf => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:23.278402 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 47/94] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:06:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 14:06:23.283609 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e1e2e6-f21f-4538-8186-450055c515b6 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:23.284447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f0b663-fad6-4e0b-a744-5a65fef210e7 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:23.388657 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:23.410211 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e1e2e6-f21f-4538-8186-450055c515b6 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:23.422134 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f0b663-fad6-4e0b-a744-5a65fef210e7 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:23.454895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567958e7-5d2c-4ab9-a24a-16bd17c22320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:23.455671 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567958e7-5d2c-4ab9-a24a-16bd17c22320 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:23.456048 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567958e7-5d2c-4ab9-a24a-16bd17c22320 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:23.456474 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567958e7-5d2c-4ab9-a24a-16bd17c22320 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:23.457397 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567958e7-5d2c-4ab9-a24a-16bd17c22320 None admin] RBAC: Adding query filter params (role.id=299c794c7acb4a5ca36288c3191aad10, scope.project.id=580a155c67b742b9bc398f50556f26cf, user.id=5e3fbbd9c18b4be39ebc01f40d1df40f) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:23.457819 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567958e7-5d2c-4ab9-a24a-16bd17c22320 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:23.459394 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567958e7-5d2c-4ab9-a24a-16bd17c22320 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:23.467341 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 48/95] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:06:23 2022] GET /identity/v3/role_assignments?user.id=5e3fbbd9c18b4be39ebc01f40d1df40f&scope.project.id=580a155c67b742b9bc398f50556f26cf&role.id=299c794c7acb4a5ca36288c3191aad10 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:23.470125 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9757710-5ac0-4fb2-b807-089398d3bf12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:23.471309 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9757710-5ac0-4fb2-b807-089398d3bf12 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:23.471738 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9757710-5ac0-4fb2-b807-089398d3bf12 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:23.472188 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9757710-5ac0-4fb2-b807-089398d3bf12 None admin] PATH_INFO: `/v3/projects/f3df77a838364810aa3aa2a2a5bbf04e` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:23.473653 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9757710-5ac0-4fb2-b807-089398d3bf12 None admin] RBAC: Authorizing `identity:get_project(project_id=f3df77a838364810aa3aa2a2a5bbf04e)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:23.475294 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9757710-5ac0-4fb2-b807-089398d3bf12 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:23.477918 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 48/96] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:06:23 2022] GET /identity/v3/projects/f3df77a838364810aa3aa2a2a5bbf04e => generated 312 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:06:23.484210 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567958e7-5d2c-4ab9-a24a-16bd17c22320 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:23.628582 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567958e7-5d2c-4ab9-a24a-16bd17c22320 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:23.668356 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b33c3e-7988-45b2-9ea2-bf6a23ae3ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:23.668933 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b33c3e-7988-45b2-9ea2-bf6a23ae3ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:23.669213 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b33c3e-7988-45b2-9ea2-bf6a23ae3ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:23.669483 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b33c3e-7988-45b2-9ea2-bf6a23ae3ea6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:23.670280 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b33c3e-7988-45b2-9ea2-bf6a23ae3ea6 None admin] RBAC: Adding query filter params (role.id=29967df108484a30a4c006cde2ae2940, scope.project.id=f3df77a838364810aa3aa2a2a5bbf04e, user.id=da4b0c24aa284fd59788ab74c424af63) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:23.670588 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b33c3e-7988-45b2-9ea2-bf6a23ae3ea6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:23.672048 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b33c3e-7988-45b2-9ea2-bf6a23ae3ea6 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:23.679692 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 49/97] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:06:23 2022] GET /identity/v3/role_assignments?user.id=da4b0c24aa284fd59788ab74c424af63&scope.project.id=f3df77a838364810aa3aa2a2a5bbf04e&role.id=29967df108484a30a4c006cde2ae2940 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:24.786223 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a2668-9c41-44c6-837f-7ac88aee5ab1 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:24.787217 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a2668-9c41-44c6-837f-7ac88aee5ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:24.787701 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a2668-9c41-44c6-837f-7ac88aee5ab1 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:24.789409 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 49/98] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:24.794591 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cefd49-1b1c-47ab-b142-f1f08b626194 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:24.795389 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cefd49-1b1c-47ab-b142-f1f08b626194 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:24.795863 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cefd49-1b1c-47ab-b142-f1f08b626194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:24.818789 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-44cefd49-1b1c-47ab-b142-f1f08b626194 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:24.820550 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-44cefd49-1b1c-47ab-b142-f1f08b626194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:24.822828 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:24.900448 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 50/99] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:24.935216 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a2668-9c41-44c6-837f-7ac88aee5ab1 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:24.966271 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:25.048781 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a2668-9c41-44c6-837f-7ac88aee5ab1 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:25.086817 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35bb81e-cb59-4211-902a-9718bb605210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:25.087470 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35bb81e-cb59-4211-902a-9718bb605210 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:25.087751 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35bb81e-cb59-4211-902a-9718bb605210 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:25.088028 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35bb81e-cb59-4211-902a-9718bb605210 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:25.099198 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35bb81e-cb59-4211-902a-9718bb605210 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:25.104367 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-b35bb81e-cb59-4211-902a-9718bb605210 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 14:06:25.104367 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:25.104367 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:25.104367 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:25.104367 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:25.104367 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:25.104367 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:25.104367 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:25.104367 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:25.104367 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:25.104367 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:25.104367 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:25.104367 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:25.104367 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:25.104367 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:25.104367 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:25.105226 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:25.105226 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:25.105226 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:25.105226 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:25.105226 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:25.105226 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:25.105226 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:25.105226 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:25.105226 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:25.105226 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:25.105226 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:25.105226 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:25.105226 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:25.105226 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:25.105226 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:25.105226 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:25.106388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:25.106388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:25.106388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:25.106388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:25.106388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:25.106388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:25.106388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:25.106388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:25.106388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:25.106388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:25.106388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 14:06:25.106388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:06:25.106388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 50/100] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:06:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:06:25.111411 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cefd49-1b1c-47ab-b142-f1f08b626194 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:25.246476 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cefd49-1b1c-47ab-b142-f1f08b626194 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:25.282875 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb31aac-82d0-488d-88fd-fd56abffa4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:25.283519 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb31aac-82d0-488d-88fd-fd56abffa4ce None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:25.283801 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb31aac-82d0-488d-88fd-fd56abffa4ce None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:25.284073 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb31aac-82d0-488d-88fd-fd56abffa4ce None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:25.284823 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb31aac-82d0-488d-88fd-fd56abffa4ce None admin] RBAC: Adding query filter params (name=member) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:25.285130 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb31aac-82d0-488d-88fd-fd56abffa4ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:25.286524 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb31aac-82d0-488d-88fd-fd56abffa4ce None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:25.295470 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 51/101] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Aug 5 14:06:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:25.296759 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:25.300903 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35bb81e-cb59-4211-902a-9718bb605210 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:25.395671 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:25.414667 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35bb81e-cb59-4211-902a-9718bb605210 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:25.448864 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a95c4b-c15a-45ea-a8cb-4a70515c9b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:25.449511 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a95c4b-c15a-45ea-a8cb-4a70515c9b9c None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:25.449846 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a95c4b-c15a-45ea-a8cb-4a70515c9b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:25.450293 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a95c4b-c15a-45ea-a8cb-4a70515c9b9c None admin] PATH_INFO: `/v3/users/da4b0c24aa284fd59788ab74c424af63` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:25.451482 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a95c4b-c15a-45ea-a8cb-4a70515c9b9c None admin] RBAC: Authorizing `identity:get_user(user_id=da4b0c24aa284fd59788ab74c424af63)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:25.452975 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a95c4b-c15a-45ea-a8cb-4a70515c9b9c None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:25.454598 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 51/102] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:06:25 2022] GET /identity/v3/users/da4b0c24aa284fd59788ab74c424af63 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:25.460774 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb31aac-82d0-488d-88fd-fd56abffa4ce None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:25.583943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb31aac-82d0-488d-88fd-fd56abffa4ce None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:25.630528 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae0ad0-114b-45cc-9d4d-037dfaa13e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:25.631179 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae0ad0-114b-45cc-9d4d-037dfaa13e89 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:25.631466 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae0ad0-114b-45cc-9d4d-037dfaa13e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:25.631853 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae0ad0-114b-45cc-9d4d-037dfaa13e89 None admin] PATH_INFO: `/v3/projects/580a155c67b742b9bc398f50556f26cf` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:25.633102 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ae0ad0-114b-45cc-9d4d-037dfaa13e89 None admin] RBAC: Authorizing `identity:get_project(project_id=580a155c67b742b9bc398f50556f26cf)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:25.634535 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ae0ad0-114b-45cc-9d4d-037dfaa13e89 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:25.636283 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 52/103] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:06:25 2022] GET /identity/v3/projects/580a155c67b742b9bc398f50556f26cf => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:25.642243 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a95c4b-c15a-45ea-a8cb-4a70515c9b9c None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:25.770160 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a95c4b-c15a-45ea-a8cb-4a70515c9b9c None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:25.814755 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd571c-d689-43f5-8126-9b8e291ace0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:25.815530 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd571c-d689-43f5-8126-9b8e291ace0c None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:25.815724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd571c-d689-43f5-8126-9b8e291ace0c None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:25.816002 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd571c-d689-43f5-8126-9b8e291ace0c None admin] PATH_INFO: `/v3/projects/580a155c67b742b9bc398f50556f26cf/users/da4b0c24aa284fd59788ab74c424af63/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:25.818023 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd571c-d689-43f5-8126-9b8e291ace0c None admin] RBAC: Authorizing `identity:create_grant(project_id=580a155c67b742b9bc398f50556f26cf, user_id=da4b0c24aa284fd59788ab74c424af63, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:25.819767 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd571c-d689-43f5-8126-9b8e291ace0c None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:25.917042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 52/104] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Aug 5 14:06:25 2022] PUT /identity/v3/projects/580a155c67b742b9bc398f50556f26cf/users/da4b0c24aa284fd59788ab74c424af63/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:06:26.212769 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09fce2-43e8-4457-aedf-ccbc0b0790eb None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:26.213738 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09fce2-43e8-4457-aedf-ccbc0b0790eb None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:26.214239 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09fce2-43e8-4457-aedf-ccbc0b0790eb None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:26.215850 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 53/105] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:26.232107 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec01150-47c1-485d-969b-88677408ff73 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:26.232396 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec01150-47c1-485d-969b-88677408ff73 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:26.232695 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec01150-47c1-485d-969b-88677408ff73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:26.254089 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-eec01150-47c1-485d-969b-88677408ff73 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:26.255298 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-eec01150-47c1-485d-969b-88677408ff73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:26.258444 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:26.274329 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26379613-6b23-4092-b4ff-913f8e1e3dc2 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:26.274617 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26379613-6b23-4092-b4ff-913f8e1e3dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:26.274899 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26379613-6b23-4092-b4ff-913f8e1e3dc2 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:26.276244 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 54/106] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:06:26.285791 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b8e70-d3be-4f41-a0df-0aa325f65950 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:26.286113 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b8e70-d3be-4f41-a0df-0aa325f65950 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:26.286362 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b8e70-d3be-4f41-a0df-0aa325f65950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:26.317055 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-954b8e70-d3be-4f41-a0df-0aa325f65950 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:26.318269 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-954b8e70-d3be-4f41-a0df-0aa325f65950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:26.373309 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 53/107] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:26.374844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff5458-7316-4ba5-8782-56595fa55ba0 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:26.375131 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff5458-7316-4ba5-8782-56595fa55ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:26.375406 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff5458-7316-4ba5-8782-56595fa55ba0 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:26.376788 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 54/108] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:26.385873 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e0ce5-1d1f-4372-807a-dcda621e621f None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:26.386163 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e0ce5-1d1f-4372-807a-dcda621e621f None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:26.386445 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e0ce5-1d1f-4372-807a-dcda621e621f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:26.412480 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:26.416629 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-dd6e0ce5-1d1f-4372-807a-dcda621e621f None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:26.417833 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-dd6e0ce5-1d1f-4372-807a-dcda621e621f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:26.425547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 55/109] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 14:06:26.426531 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b8e70-d3be-4f41-a0df-0aa325f65950 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:26.487612 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 55/110] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:26.492833 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e0ce5-1d1f-4372-807a-dcda621e621f None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:26.629127 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b8e70-d3be-4f41-a0df-0aa325f65950 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:26.646689 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:26.671949 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c315f-cdc7-4ba2-ade0-0eab874e239c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:26.672877 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c315f-cdc7-4ba2-ade0-0eab874e239c None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:26.673303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c315f-cdc7-4ba2-ade0-0eab874e239c None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:26.673745 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c315f-cdc7-4ba2-ade0-0eab874e239c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:26.681143 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e0ce5-1d1f-4372-807a-dcda621e621f None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:26.685096 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21c315f-cdc7-4ba2-ade0-0eab874e239c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:26.691526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-b21c315f-cdc7-4ba2-ade0-0eab874e239c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 14:06:26.691526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:26.691526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:26.691526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:26.691526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:26.691526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:26.691526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:26.691526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:26.691526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:26.691526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:26.691526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:26.691526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:26.691526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:26.691526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:26.691526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:26.691526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:26.692609 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:26.692609 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:26.692609 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:26.692609 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:26.692609 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:26.692609 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:26.692609 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:26.692609 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:26.692609 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:26.692609 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:26.692609 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:26.692609 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:26.692609 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:26.692609 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:26.692609 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:26.692609 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:26.693549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:26.693549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:26.693549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:26.693549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:26.693549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:26.693549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:26.693549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:26.693549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:26.693549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:26.693549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:26.693549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 14:06:26.693549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:06:26.693549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 56/111] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:06:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:06:26.693549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c315f-cdc7-4ba2-ade0-0eab874e239c None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:26.722301 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e35325-ceaa-4e0d-af7f-35a4f4f42690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:26.723284 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e35325-ceaa-4e0d-af7f-35a4f4f42690 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:26.723708 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e35325-ceaa-4e0d-af7f-35a4f4f42690 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:26.724134 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e35325-ceaa-4e0d-af7f-35a4f4f42690 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:26.735748 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e35325-ceaa-4e0d-af7f-35a4f4f42690 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:26.743285 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-51e35325-ceaa-4e0d-af7f-35a4f4f42690 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 14:06:26.743285 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:26.743285 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:26.743285 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:26.743285 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:26.743285 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:26.743285 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:26.743285 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:26.743285 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:26.743285 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:26.743285 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:26.743285 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:26.743285 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:26.743285 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:26.743285 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:26.743285 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:26.744326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:26.744326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:26.744326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:26.744326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:26.744326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:26.744326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:26.744326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:26.744326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:26.744326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:26.744326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:26.744326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:26.744326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:26.744326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:26.744326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:26.744326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:26.744326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:26.745107 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:26.745107 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:26.745107 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:26.745107 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:26.745107 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:26.745107 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:26.745107 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:26.745107 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:26.745107 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:26.745107 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:26.745107 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 14:06:26.745107 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:06:26.747641 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 56/112] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Aug 5 14:06:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:06:26.749315 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e066e2-d61a-48ef-8155-6a4ea696a10c None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:26.749774 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e066e2-d61a-48ef-8155-6a4ea696a10c None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:26.750222 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e066e2-d61a-48ef-8155-6a4ea696a10c None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:26.751624 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 57/113] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:26.752498 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e066e2-d61a-48ef-8155-6a4ea696a10c None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:26.766675 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:26.813070 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c315f-cdc7-4ba2-ade0-0eab874e239c None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:26.851968 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b4015-e440-4689-bf14-b8a9eede9c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:26.852740 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b4015-e440-4689-bf14-b8a9eede9c95 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:26.853183 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b4015-e440-4689-bf14-b8a9eede9c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:26.853633 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b4015-e440-4689-bf14-b8a9eede9c95 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:26.854947 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84b4015-e440-4689-bf14-b8a9eede9c95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:26.856730 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84b4015-e440-4689-bf14-b8a9eede9c95 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:26.858757 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 57/114] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:06:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:26.860082 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a868e-3aa4-4133-8b38-2af458409e76 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:26.860406 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e066e2-d61a-48ef-8155-6a4ea696a10c None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:26.861006 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a868e-3aa4-4133-8b38-2af458409e76 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:26.861409 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a868e-3aa4-4133-8b38-2af458409e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:26.879487 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-ef0a868e-3aa4-4133-8b38-2af458409e76 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:26.880735 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-ef0a868e-3aa4-4133-8b38-2af458409e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:26.892364 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:26.898572 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e6af8-7139-4dd3-86ad-89aa3e6589a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:26.899197 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e6af8-7139-4dd3-86ad-89aa3e6589a0 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:26.899476 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e6af8-7139-4dd3-86ad-89aa3e6589a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:26.899749 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e6af8-7139-4dd3-86ad-89aa3e6589a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:26.900495 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e6af8-7139-4dd3-86ad-89aa3e6589a0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:26.906162 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e6af8-7139-4dd3-86ad-89aa3e6589a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:26.906162 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e6af8-7139-4dd3-86ad-89aa3e6589a0 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:26.910487 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 58/115] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Aug 5 14:06:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:26.911119 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e6af8-7139-4dd3-86ad-89aa3e6589a0 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:26.959002 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 58/116] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:26.960855 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de727e31-5467-4cdc-b88d-d90c5eb2fda8 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:26.961040 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de727e31-5467-4cdc-b88d-d90c5eb2fda8 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:26.961318 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de727e31-5467-4cdc-b88d-d90c5eb2fda8 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:26.962740 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 59/117] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:26.965150 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de727e31-5467-4cdc-b88d-d90c5eb2fda8 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:27.026340 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:27.038006 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e6af8-7139-4dd3-86ad-89aa3e6589a0 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:27.072192 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de727e31-5467-4cdc-b88d-d90c5eb2fda8 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:27.077508 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd8f8d-6b4c-44cd-b62f-65632c11a186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:27.078129 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd8f8d-6b4c-44cd-b62f-65632c11a186 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:27.078412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd8f8d-6b4c-44cd-b62f-65632c11a186 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:27.078688 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd8f8d-6b4c-44cd-b62f-65632c11a186 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:27.079403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fd8f8d-6b4c-44cd-b62f-65632c11a186 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:27.079711 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fd8f8d-6b4c-44cd-b62f-65632c11a186 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:27.081544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fd8f8d-6b4c-44cd-b62f-65632c11a186 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:27.091579 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 59/118] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:06:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:27.096824 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd8f8d-6b4c-44cd-b62f-65632c11a186 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:27.109963 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01931440-b224-4f20-808c-3b2615d4559e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:27.110503 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01931440-b224-4f20-808c-3b2615d4559e None admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:27.110783 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01931440-b224-4f20-808c-3b2615d4559e None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:27.111061 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01931440-b224-4f20-808c-3b2615d4559e None admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:27.111735 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01931440-b224-4f20-808c-3b2615d4559e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:27.113305 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01931440-b224-4f20-808c-3b2615d4559e None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:27.153646 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:27.191341 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 60/119] 10.222.0.26 () {60 vars in 1141 bytes} [Fri Aug 5 14:06:26 2022] POST /identity/v3/projects => generated 302 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:06:27.197888 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd8f8d-6b4c-44cd-b62f-65632c11a186 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:27.201993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e87034-57a6-48a9-96ed-a14d1d348338 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:27.202284 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e87034-57a6-48a9-96ed-a14d1d348338 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:27.202567 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e87034-57a6-48a9-96ed-a14d1d348338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:27.239423 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-27e87034-57a6-48a9-96ed-a14d1d348338 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:27.241532 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae6eedb-5210-425d-9c9f-f9d301e30091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:27.242258 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6eedb-5210-425d-9c9f-f9d301e30091 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:27.242538 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6eedb-5210-425d-9c9f-f9d301e30091 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:27.242809 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6eedb-5210-425d-9c9f-f9d301e30091 None admin] PATH_INFO: `/v3/users/f62da411608041ce960346ac7faf6bbd` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:27.243263 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-27e87034-57a6-48a9-96ed-a14d1d348338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:27.261755 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae6eedb-5210-425d-9c9f-f9d301e30091 None admin] RBAC: Authorizing `identity:get_user(user_id=f62da411608041ce960346ac7faf6bbd)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:27.263407 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae6eedb-5210-425d-9c9f-f9d301e30091 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:27.265451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 60/120] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:06:27 2022] GET /identity/v3/users/f62da411608041ce960346ac7faf6bbd => generated 289 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:27.265451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae6eedb-5210-425d-9c9f-f9d301e30091 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:27.330789 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 61/121] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:27.331881 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e87034-57a6-48a9-96ed-a14d1d348338 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:27.398886 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:27.414903 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae6eedb-5210-425d-9c9f-f9d301e30091 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:27.434128 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:27.464196 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df97f8c-c691-41b7-a69b-3d5d5a253ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:27.464886 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df97f8c-c691-41b7-a69b-3d5d5a253ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:27.465170 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df97f8c-c691-41b7-a69b-3d5d5a253ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:27.465439 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df97f8c-c691-41b7-a69b-3d5d5a253ec0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:27.480675 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df97f8c-c691-41b7-a69b-3d5d5a253ec0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:27.481721 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e87034-57a6-48a9-96ed-a14d1d348338 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:27.487168 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-4df97f8c-c691-41b7-a69b-3d5d5a253ec0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:06:27.487168 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:27.487168 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:27.487168 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:27.487168 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:27.487168 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:27.487168 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:27.487168 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:27.487168 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:27.487168 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:27.487168 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:27.487168 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:27.487168 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:27.487168 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:27.487168 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:27.487168 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:27.488046 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:27.488046 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:27.488046 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:27.488046 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:27.488046 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:27.488046 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:27.488046 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:27.488046 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:27.488046 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:27.488046 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:27.488046 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:27.488046 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:27.488046 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:27.488046 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:27.488046 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:27.488046 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:27.489187 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:27.489187 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:27.489187 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:27.489187 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:27.489187 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:27.489187 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:27.489187 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:27.489187 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:27.489187 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:27.489187 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:27.489187 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:06:27.489187 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:06:27.489187 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 61/122] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Aug 5 14:06:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:06:27.489847 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df97f8c-c691-41b7-a69b-3d5d5a253ec0 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:27.544004 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a028205-a60f-4db4-a29d-8e94ac86bed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:27.544694 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a028205-a60f-4db4-a29d-8e94ac86bed2 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:27.564839 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a028205-a60f-4db4-a29d-8e94ac86bed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:27.564839 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a028205-a60f-4db4-a29d-8e94ac86bed2 None admin] PATH_INFO: `/v3/users/da4b0c24aa284fd59788ab74c424af63` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:27.564839 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a028205-a60f-4db4-a29d-8e94ac86bed2 None admin] RBAC: Authorizing `identity:get_user(user_id=da4b0c24aa284fd59788ab74c424af63)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:27.564839 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a028205-a60f-4db4-a29d-8e94ac86bed2 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:27.564839 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 62/123] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:06:27 2022] GET /identity/v3/users/da4b0c24aa284fd59788ab74c424af63 => generated 275 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:27.564839 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a028205-a60f-4db4-a29d-8e94ac86bed2 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:27.613762 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df97f8c-c691-41b7-a69b-3d5d5a253ec0 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:27.690821 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a028205-a60f-4db4-a29d-8e94ac86bed2 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:27.697269 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0377b8-20b4-42b0-9538-9a6bfdd7ae81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:27.698173 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0377b8-20b4-42b0-9538-9a6bfdd7ae81 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:27.698699 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0377b8-20b4-42b0-9538-9a6bfdd7ae81 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:27.699132 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:27.699919 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0377b8-20b4-42b0-9538-9a6bfdd7ae81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:27.728271 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2ff5bc-bb5e-47e2-beb5-4b5afeb4e2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:27.729104 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ff5bc-bb5e-47e2-beb5-4b5afeb4e2fb None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:27.729393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ff5bc-bb5e-47e2-beb5-4b5afeb4e2fb None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:27.729674 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ff5bc-bb5e-47e2-beb5-4b5afeb4e2fb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:27.742054 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2ff5bc-bb5e-47e2-beb5-4b5afeb4e2fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:27.746581 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-fe2ff5bc-bb5e-47e2-beb5-4b5afeb4e2fb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 14:06:27.746581 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:27.746581 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:27.746581 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:27.746581 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:27.746581 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:27.746581 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:27.746581 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:27.746581 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:27.746581 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:27.746581 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:27.746581 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:27.746581 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:27.746581 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:27.746581 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:27.746581 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:27.747464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:27.747464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:27.747464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:27.747464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:27.747464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:27.747464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:27.747464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:27.747464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:27.747464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:27.747464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:27.747464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:27.747464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:27.747464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:27.747464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:27.747464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:27.747464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:27.748262 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:27.748262 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:27.748262 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:27.748262 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:27.748262 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:27.748262 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:27.748262 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:27.748262 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:27.748262 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:27.748262 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:27.748262 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 14:06:27.748262 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:06:27.748262 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 63/124] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:06:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:06:27.752055 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:27.753302 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2ff5bc-bb5e-47e2-beb5-4b5afeb4e2fb None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:27.782885 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0377b8-20b4-42b0-9538-9a6bfdd7ae81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:27.784869 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0377b8-20b4-42b0-9538-9a6bfdd7ae81 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:27.861531 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2ff5bc-bb5e-47e2-beb5-4b5afeb4e2fb None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:27.902833 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 62/125] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:06:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 14:06:27.902833 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0377b8-20b4-42b0-9538-9a6bfdd7ae81 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:27.905084 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a5642e-bacb-49f4-a406-40910f937d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:27.905719 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a5642e-bacb-49f4-a406-40910f937d51 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:27.906005 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a5642e-bacb-49f4-a406-40910f937d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:27.906282 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a5642e-bacb-49f4-a406-40910f937d51 None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:27.907008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a5642e-bacb-49f4-a406-40910f937d51 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:27.907315 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a5642e-bacb-49f4-a406-40910f937d51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:27.908736 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a5642e-bacb-49f4-a406-40910f937d51 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:27.918213 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 64/126] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:06:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:27.918884 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a5642e-bacb-49f4-a406-40910f937d51 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:27.948650 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:27.985130 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:28.020520 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0377b8-20b4-42b0-9538-9a6bfdd7ae81 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:28.025648 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a5642e-bacb-49f4-a406-40910f937d51 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:28.057961 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6049c89e-4617-49b6-9032-bfb24dd539e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:28.059484 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049c89e-4617-49b6-9032-bfb24dd539e5 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:28.060056 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049c89e-4617-49b6-9032-bfb24dd539e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:28.060687 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049c89e-4617-49b6-9032-bfb24dd539e5 None admin] PATH_INFO: `/v3/projects/580a155c67b742b9bc398f50556f26cf` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:28.061876 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3556657f-1459-4ac2-9347-cb5ad56801ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:28.062265 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6049c89e-4617-49b6-9032-bfb24dd539e5 None admin] RBAC: Authorizing `identity:get_project(project_id=580a155c67b742b9bc398f50556f26cf)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:28.062715 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556657f-1459-4ac2-9347-cb5ad56801ba None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:28.063099 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556657f-1459-4ac2-9347-cb5ad56801ba None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:28.063485 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556657f-1459-4ac2-9347-cb5ad56801ba None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:28.064301 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3556657f-1459-4ac2-9347-cb5ad56801ba None admin] RBAC: Adding query filter params (name=member) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:28.064497 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6049c89e-4617-49b6-9032-bfb24dd539e5 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:28.064817 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3556657f-1459-4ac2-9347-cb5ad56801ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:28.066274 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3556657f-1459-4ac2-9347-cb5ad56801ba None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:28.066744 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 63/127] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:06:27 2022] GET /identity/v3/projects/580a155c67b742b9bc398f50556f26cf => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:28.067660 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6049c89e-4617-49b6-9032-bfb24dd539e5 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:28.076525 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 65/128] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Aug 5 14:06:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:28.077475 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3556657f-1459-4ac2-9347-cb5ad56801ba None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:28.177316 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6049c89e-4617-49b6-9032-bfb24dd539e5 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:28.179921 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:28.190049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3556657f-1459-4ac2-9347-cb5ad56801ba None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:28.215132 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e266b42-cc80-4c9e-9754-319bdd976a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:28.215998 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e266b42-cc80-4c9e-9754-319bdd976a50 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:28.216504 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e266b42-cc80-4c9e-9754-319bdd976a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:28.217000 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e266b42-cc80-4c9e-9754-319bdd976a50 None admin] PATH_INFO: `/v3/projects/580a155c67b742b9bc398f50556f26cf` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:28.218384 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e266b42-cc80-4c9e-9754-319bdd976a50 None admin] RBAC: Authorizing `identity:get_project(project_id=580a155c67b742b9bc398f50556f26cf)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:28.220059 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e266b42-cc80-4c9e-9754-319bdd976a50 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:28.226066 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 64/129] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:06:28 2022] GET /identity/v3/projects/580a155c67b742b9bc398f50556f26cf => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:28.226066 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e266b42-cc80-4c9e-9754-319bdd976a50 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:28.237188 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb9631-1072-4a54-8869-dd80cc8a84ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:28.238065 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb9631-1072-4a54-8869-dd80cc8a84ca None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:28.238507 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb9631-1072-4a54-8869-dd80cc8a84ca None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:28.239434 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb9631-1072-4a54-8869-dd80cc8a84ca None admin] PATH_INFO: `/v3/users/5e3fbbd9c18b4be39ebc01f40d1df40f` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:28.242220 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:28.242865 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeb9631-1072-4a54-8869-dd80cc8a84ca None admin] RBAC: Authorizing `identity:get_user(user_id=5e3fbbd9c18b4be39ebc01f40d1df40f)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:28.245418 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeb9631-1072-4a54-8869-dd80cc8a84ca None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:28.248212 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 66/130] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:06:28 2022] GET /identity/v3/users/5e3fbbd9c18b4be39ebc01f40d1df40f => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:28.249684 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb9631-1072-4a54-8869-dd80cc8a84ca None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:28.354745 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e266b42-cc80-4c9e-9754-319bdd976a50 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:28.373853 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb9631-1072-4a54-8869-dd80cc8a84ca None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:28.397557 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f7609-e616-46c6-97d4-bcf5d2c183b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:28.398232 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f7609-e616-46c6-97d4-bcf5d2c183b8 None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:28.398518 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f7609-e616-46c6-97d4-bcf5d2c183b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:28.398786 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f7609-e616-46c6-97d4-bcf5d2c183b8 None admin] PATH_INFO: `/v3/projects/580a155c67b742b9bc398f50556f26cf/users/da4b0c24aa284fd59788ab74c424af63/roles/199fd3093a824ba585b1135c8bf363be` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:28.400979 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1f7609-e616-46c6-97d4-bcf5d2c183b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=580a155c67b742b9bc398f50556f26cf, user_id=da4b0c24aa284fd59788ab74c424af63, role_id=199fd3093a824ba585b1135c8bf363be)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:28.402409 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1f7609-e616-46c6-97d4-bcf5d2c183b8 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:28.410565 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9066ab9-6537-445d-a0bd-7673e66e135e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:28.411210 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9066ab9-6537-445d-a0bd-7673e66e135e None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:28.411490 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9066ab9-6537-445d-a0bd-7673e66e135e None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:28.411758 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9066ab9-6537-445d-a0bd-7673e66e135e None admin] PATH_INFO: `/v3/users/da4b0c24aa284fd59788ab74c424af63` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:28.413601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9066ab9-6537-445d-a0bd-7673e66e135e None admin] RBAC: Authorizing `identity:get_user(user_id=da4b0c24aa284fd59788ab74c424af63)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:28.417891 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9066ab9-6537-445d-a0bd-7673e66e135e None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:28.417891 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 67/131] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:06:28 2022] GET /identity/v3/users/da4b0c24aa284fd59788ab74c424af63 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:28.417891 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9066ab9-6537-445d-a0bd-7673e66e135e None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:28.445764 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:28.508444 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 65/132] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Aug 5 14:06:28 2022] PUT /identity/v3/projects/580a155c67b742b9bc398f50556f26cf/users/da4b0c24aa284fd59788ab74c424af63/roles/199fd3093a824ba585b1135c8bf363be => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:06:28.508444 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f7609-e616-46c6-97d4-bcf5d2c183b8 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:28.549215 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9066ab9-6537-445d-a0bd-7673e66e135e None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:28.588339 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee5005-1e30-4f65-94ba-15e63487eab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:28.589021 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee5005-1e30-4f65-94ba-15e63487eab8 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:28.589349 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee5005-1e30-4f65-94ba-15e63487eab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:28.589624 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee5005-1e30-4f65-94ba-15e63487eab8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:28.590436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee5005-1e30-4f65-94ba-15e63487eab8 None admin] RBAC: Adding query filter params (role.id=8e4d742669e54e288bac0e45718908b8, scope.project.id=580a155c67b742b9bc398f50556f26cf, user.id=f62da411608041ce960346ac7faf6bbd) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:28.590801 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee5005-1e30-4f65-94ba-15e63487eab8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:28.592471 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee5005-1e30-4f65-94ba-15e63487eab8 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:28.599147 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 68/133] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:06:28 2022] GET /identity/v3/role_assignments?user.id=f62da411608041ce960346ac7faf6bbd&scope.project.id=580a155c67b742b9bc398f50556f26cf&role.id=8e4d742669e54e288bac0e45718908b8 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:28.606362 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148d249-bf4f-47dd-a3ce-3365aef10a88 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:28.606626 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148d249-bf4f-47dd-a3ce-3365aef10a88 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:28.613975 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148d249-bf4f-47dd-a3ce-3365aef10a88 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:28.613975 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 69/134] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:28 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:28.613975 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148d249-bf4f-47dd-a3ce-3365aef10a88 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:28.614376 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:28.687137 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f7609-e616-46c6-97d4-bcf5d2c183b8 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:28.743940 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc6598-07f4-4917-8a4e-f0c77a68b898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:28.744612 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc6598-07f4-4917-8a4e-f0c77a68b898 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:28.744899 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc6598-07f4-4917-8a4e-f0c77a68b898 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:28.745176 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc6598-07f4-4917-8a4e-f0c77a68b898 None admin] PATH_INFO: `/v3/projects/580a155c67b742b9bc398f50556f26cf` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:28.746419 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc6598-07f4-4917-8a4e-f0c77a68b898 None admin] RBAC: Authorizing `identity:get_project(project_id=580a155c67b742b9bc398f50556f26cf)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:28.747905 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc6598-07f4-4917-8a4e-f0c77a68b898 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:28.750221 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 66/135] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:06:28 2022] GET /identity/v3/projects/580a155c67b742b9bc398f50556f26cf => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:28.751407 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9afc6f7-5b11-4484-87c6-1007be42fc76 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:28.751704 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9afc6f7-5b11-4484-87c6-1007be42fc76 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:28.751985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9afc6f7-5b11-4484-87c6-1007be42fc76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:28.767646 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:28.779253 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148d249-bf4f-47dd-a3ce-3365aef10a88 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:28.780289 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-a9afc6f7-5b11-4484-87c6-1007be42fc76 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:28.781603 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a9afc6f7-5b11-4484-87c6-1007be42fc76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:28.816646 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9673a30-1483-4856-8617-a1cc86c3fbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:28.817296 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9673a30-1483-4856-8617-a1cc86c3fbe3 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:28.817579 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9673a30-1483-4856-8617-a1cc86c3fbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:28.818963 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9673a30-1483-4856-8617-a1cc86c3fbe3 None admin] PATH_INFO: `/v3/projects/f3df77a838364810aa3aa2a2a5bbf04e` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:28.821122 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9673a30-1483-4856-8617-a1cc86c3fbe3 None admin] RBAC: Authorizing `identity:get_project(project_id=f3df77a838364810aa3aa2a2a5bbf04e)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:28.822584 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9673a30-1483-4856-8617-a1cc86c3fbe3 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:28.824454 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 70/136] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:06:28 2022] GET /identity/v3/projects/f3df77a838364810aa3aa2a2a5bbf04e => generated 312 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:28.829827 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9673a30-1483-4856-8617-a1cc86c3fbe3 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:28.867318 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 67/137] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:28.868162 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9afc6f7-5b11-4484-87c6-1007be42fc76 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:28.937712 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9673a30-1483-4856-8617-a1cc86c3fbe3 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:28.940785 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:28.974161 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61f15f-e8c3-42df-a501-32004c92caaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:28.974908 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61f15f-e8c3-42df-a501-32004c92caaf None admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:28.975260 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61f15f-e8c3-42df-a501-32004c92caaf None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:28.975541 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61f15f-e8c3-42df-a501-32004c92caaf None admin] PATH_INFO: `/v3/projects/580a155c67b742b9bc398f50556f26cf/users/5e3fbbd9c18b4be39ebc01f40d1df40f/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:28.977704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec61f15f-e8c3-42df-a501-32004c92caaf None admin] RBAC: Authorizing `identity:create_grant(project_id=580a155c67b742b9bc398f50556f26cf, user_id=5e3fbbd9c18b4be39ebc01f40d1df40f, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:28.979145 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec61f15f-e8c3-42df-a501-32004c92caaf None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:28.983115 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9afc6f7-5b11-4484-87c6-1007be42fc76 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:29.024649 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92439e4-fbbf-481b-b74e-28bffbed36af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:29.025288 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92439e4-fbbf-481b-b74e-28bffbed36af None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:29.025628 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92439e4-fbbf-481b-b74e-28bffbed36af None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:29.025899 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92439e4-fbbf-481b-b74e-28bffbed36af None admin] PATH_INFO: `/v3/projects/f3df77a838364810aa3aa2a2a5bbf04e/users/da4b0c24aa284fd59788ab74c424af63/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:29.027874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92439e4-fbbf-481b-b74e-28bffbed36af None admin] RBAC: Authorizing `identity:create_grant(project_id=f3df77a838364810aa3aa2a2a5bbf04e, user_id=da4b0c24aa284fd59788ab74c424af63, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:29.029429 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92439e4-fbbf-481b-b74e-28bffbed36af None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:29.079105 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 71/138] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Aug 5 14:06:28 2022] PUT /identity/v3/projects/580a155c67b742b9bc398f50556f26cf/users/5e3fbbd9c18b4be39ebc01f40d1df40f/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:06:29.080030 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61f15f-e8c3-42df-a501-32004c92caaf None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:29.106609 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:29.142825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 68/139] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Aug 5 14:06:28 2022] PUT /identity/v3/projects/f3df77a838364810aa3aa2a2a5bbf04e/users/da4b0c24aa284fd59788ab74c424af63/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:06:29.216782 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61f15f-e8c3-42df-a501-32004c92caaf None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:29.264370 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bcbf67-6c23-46a2-b796-6eff88ed238d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:29.265898 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bcbf67-6c23-46a2-b796-6eff88ed238d None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:29.266373 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bcbf67-6c23-46a2-b796-6eff88ed238d None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:29.266838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bcbf67-6c23-46a2-b796-6eff88ed238d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:29.284430 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bcbf67-6c23-46a2-b796-6eff88ed238d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:29.290953 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-91bcbf67-6c23-46a2-b796-6eff88ed238d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 14:06:29.290953 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:29.290953 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:29.290953 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:29.290953 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:29.290953 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:29.290953 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:29.290953 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:29.290953 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:29.290953 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:29.290953 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:29.290953 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:29.290953 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:29.290953 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:29.290953 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:29.290953 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:29.291897 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:29.291897 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:29.291897 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:29.291897 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:29.291897 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:29.291897 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:29.291897 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:29.291897 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:29.291897 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:29.291897 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:29.291897 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:29.291897 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:29.291897 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:29.291897 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:29.291897 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:29.291897 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:29.292687 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:29.292687 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:29.292687 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:29.292687 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:29.292687 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:29.292687 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:29.292687 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:29.292687 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:29.292687 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:29.292687 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:29.292687 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 14:06:29.292687 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:06:29.295616 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 72/140] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:06:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:06:29.301389 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92439e4-fbbf-481b-b74e-28bffbed36af None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:29.387453 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4756a-cecf-4656-9040-ff652776601c None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:29.388081 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4756a-cecf-4656-9040-ff652776601c None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:29.388356 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4756a-cecf-4656-9040-ff652776601c None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:29.389858 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 73/141] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:06:29.395028 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db44cd73-6e04-4c5f-b618-1606b212edae None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:29.395307 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db44cd73-6e04-4c5f-b618-1606b212edae None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:29.395576 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db44cd73-6e04-4c5f-b618-1606b212edae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:29.418071 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-db44cd73-6e04-4c5f-b618-1606b212edae None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:29.419631 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-db44cd73-6e04-4c5f-b618-1606b212edae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:29.437070 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92439e4-fbbf-481b-b74e-28bffbed36af None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:29.465858 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:29.476256 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe452f-6fa3-4c00-ab9f-1d67a2c0790c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:29.476993 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe452f-6fa3-4c00-ab9f-1d67a2c0790c None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:29.477242 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe452f-6fa3-4c00-ab9f-1d67a2c0790c None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:29.477514 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe452f-6fa3-4c00-ab9f-1d67a2c0790c None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:29.478331 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe452f-6fa3-4c00-ab9f-1d67a2c0790c None admin] RBAC: Adding query filter params (name=member) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:29.478637 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe452f-6fa3-4c00-ab9f-1d67a2c0790c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:29.480212 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe452f-6fa3-4c00-ab9f-1d67a2c0790c None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:29.491519 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 69/142] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Aug 5 14:06:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:29.499073 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe452f-6fa3-4c00-ab9f-1d67a2c0790c None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:29.502584 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 74/143] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 14:06:29.529361 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db44cd73-6e04-4c5f-b618-1606b212edae None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:29.542327 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:29.639372 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe452f-6fa3-4c00-ab9f-1d67a2c0790c None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:29.645552 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db44cd73-6e04-4c5f-b618-1606b212edae None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:29.679544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17902cf2-c124-44eb-8f60-da81f00b2db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:29.680635 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17902cf2-c124-44eb-8f60-da81f00b2db2 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:29.681117 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17902cf2-c124-44eb-8f60-da81f00b2db2 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:29.681614 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17902cf2-c124-44eb-8f60-da81f00b2db2 None admin] PATH_INFO: `/v3/users/da4b0c24aa284fd59788ab74c424af63` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:29.683073 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17902cf2-c124-44eb-8f60-da81f00b2db2 None admin] RBAC: Authorizing `identity:get_user(user_id=da4b0c24aa284fd59788ab74c424af63)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:29.684802 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17902cf2-c124-44eb-8f60-da81f00b2db2 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:29.686759 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 70/144] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:06:29 2022] GET /identity/v3/users/da4b0c24aa284fd59788ab74c424af63 => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:29.691897 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef337f-d950-4425-98f4-8b6ba76f64e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:29.693023 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17902cf2-c124-44eb-8f60-da81f00b2db2 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:29.697097 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef337f-d950-4425-98f4-8b6ba76f64e3 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:29.697599 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef337f-d950-4425-98f4-8b6ba76f64e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:29.698904 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef337f-d950-4425-98f4-8b6ba76f64e3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:29.704107 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cef337f-d950-4425-98f4-8b6ba76f64e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:29.705855 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cef337f-d950-4425-98f4-8b6ba76f64e3 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:29.708921 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 75/145] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:06:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:06:29.715307 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef337f-d950-4425-98f4-8b6ba76f64e3 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:29.784151 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:29.839181 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17902cf2-c124-44eb-8f60-da81f00b2db2 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:29.862412 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:29.887766 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d49315-2524-4260-ac22-fcae02e78b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:29.888401 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d49315-2524-4260-ac22-fcae02e78b2d None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:29.889443 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d49315-2524-4260-ac22-fcae02e78b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:29.889731 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d49315-2524-4260-ac22-fcae02e78b2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:29.912227 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef337f-d950-4425-98f4-8b6ba76f64e3 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:29.974810 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fdc858-de79-4814-978d-fd4f72fef80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:29.976365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fdc858-de79-4814-978d-fd4f72fef80b None admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:29.976671 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fdc858-de79-4814-978d-fd4f72fef80b None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:29.976944 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fdc858-de79-4814-978d-fd4f72fef80b None admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:29.977649 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fdc858-de79-4814-978d-fd4f72fef80b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:29.979333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fdc858-de79-4814-978d-fd4f72fef80b None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:30.012090 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d49315-2524-4260-ac22-fcae02e78b2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:30.013664 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d49315-2524-4260-ac22-fcae02e78b2d None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:30.086469 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:30.098094 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 76/146] 10.222.0.26 () {60 vars in 1132 bytes} [Fri Aug 5 14:06:29 2022] POST /identity/v3/users => generated 283 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:06:30.158931 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 71/147] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:06:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 14:06:30.165074 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fdc858-de79-4814-978d-fd4f72fef80b None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:30.314547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fdc858-de79-4814-978d-fd4f72fef80b None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:30.356011 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:30.362204 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6804a5a5-cbd4-42f5-9014-bd4c6ee196ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:30.366702 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804a5a5-cbd4-42f5-9014-bd4c6ee196ee None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:30.366702 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804a5a5-cbd4-42f5-9014-bd4c6ee196ee None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:30.366702 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804a5a5-cbd4-42f5-9014-bd4c6ee196ee None admin] PATH_INFO: `/v3/projects/580a155c67b742b9bc398f50556f26cf` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:30.366702 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6804a5a5-cbd4-42f5-9014-bd4c6ee196ee None admin] RBAC: Authorizing `identity:get_project(project_id=580a155c67b742b9bc398f50556f26cf)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:30.366702 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6804a5a5-cbd4-42f5-9014-bd4c6ee196ee None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:30.368168 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 77/148] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:06:30 2022] GET /identity/v3/projects/580a155c67b742b9bc398f50556f26cf => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:30.374145 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d49315-2524-4260-ac22-fcae02e78b2d None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:30.489003 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d49315-2524-4260-ac22-fcae02e78b2d None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:30.531932 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a332e070-a14f-41da-be58-cc4a0bbe21f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:30.532617 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332e070-a14f-41da-be58-cc4a0bbe21f8 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:30.532938 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332e070-a14f-41da-be58-cc4a0bbe21f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:30.533174 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332e070-a14f-41da-be58-cc4a0bbe21f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:30.533967 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a332e070-a14f-41da-be58-cc4a0bbe21f8 None admin] RBAC: Adding query filter params (role.id=29967df108484a30a4c006cde2ae2940, scope.project.id=580a155c67b742b9bc398f50556f26cf, user.id=da4b0c24aa284fd59788ab74c424af63) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:30.534276 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a332e070-a14f-41da-be58-cc4a0bbe21f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:30.535720 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a332e070-a14f-41da-be58-cc4a0bbe21f8 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:30.541999 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 72/149] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:06:30 2022] GET /identity/v3/role_assignments?user.id=da4b0c24aa284fd59788ab74c424af63&scope.project.id=580a155c67b742b9bc398f50556f26cf&role.id=29967df108484a30a4c006cde2ae2940 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:31.359084 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372a6aa-5b21-43d2-89a2-2bab3cf30b2a None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:31.359585 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372a6aa-5b21-43d2-89a2-2bab3cf30b2a None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:31.359657 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372a6aa-5b21-43d2-89a2-2bab3cf30b2a None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:31.361026 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 78/150] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:31.367497 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221fb4c-181c-4ccd-abab-22bf08fb7b87 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:31.367930 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221fb4c-181c-4ccd-abab-22bf08fb7b87 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:31.368103 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221fb4c-181c-4ccd-abab-22bf08fb7b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:31.388036 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-0221fb4c-181c-4ccd-abab-22bf08fb7b87 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:31.389264 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-0221fb4c-181c-4ccd-abab-22bf08fb7b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:31.412090 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:31.484944 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 73/151] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:31.594407 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372a6aa-5b21-43d2-89a2-2bab3cf30b2a None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:31.656821 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372a6aa-5b21-43d2-89a2-2bab3cf30b2a None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:31.697768 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d434c33-2919-4b5c-8eb5-643094b3328e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:31.698423 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d434c33-2919-4b5c-8eb5-643094b3328e None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:31.698735 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d434c33-2919-4b5c-8eb5-643094b3328e None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:31.699082 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d434c33-2919-4b5c-8eb5-643094b3328e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:31.713790 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d434c33-2919-4b5c-8eb5-643094b3328e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:31.719458 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-2d434c33-2919-4b5c-8eb5-643094b3328e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 14:06:31.719458 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:31.719458 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:31.719458 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:31.719458 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:31.719458 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:31.719458 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:31.719458 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:31.719458 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:31.719458 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:31.719458 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:31.719458 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:31.719458 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:31.719458 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:31.719458 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:31.719458 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:31.720316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:31.720316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:31.720316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:31.720316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:31.720316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:31.720316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:31.720316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:31.720316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:31.720316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:31.720316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:31.720316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:31.720316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:31.720316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:31.720316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:31.720316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:31.720316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:31.721129 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:31.721129 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:31.721129 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:31.721129 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:31.721129 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:31.721129 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:31.721129 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:31.721129 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:31.721129 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:31.721129 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:31.721129 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 14:06:31.721129 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:06:31.721129 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 79/152] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:06:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:06:31.727122 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0221fb4c-181c-4ccd-abab-22bf08fb7b87 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:31.790728 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5af31-dac0-4d12-86a9-a0b430748fab None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:31.791039 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5af31-dac0-4d12-86a9-a0b430748fab None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:31.791280 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5af31-dac0-4d12-86a9-a0b430748fab None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:31.792671 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 80/153] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:06:31.798848 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f1bd7-c382-4d97-a207-a4fb40cc98e3 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:31.799134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f1bd7-c382-4d97-a207-a4fb40cc98e3 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:31.799408 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f1bd7-c382-4d97-a207-a4fb40cc98e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:31.821344 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-1a1f1bd7-c382-4d97-a207-a4fb40cc98e3 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:31.822562 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-1a1f1bd7-c382-4d97-a207-a4fb40cc98e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:31.890619 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0221fb4c-181c-4ccd-abab-22bf08fb7b87 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:31.898300 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:31.918135 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 81/154] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 14:06:31.952501 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f1bd7-c382-4d97-a207-a4fb40cc98e3 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:31.962071 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d380a4-33f5-42d8-adfc-a1cc0673a2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:31.962711 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d380a4-33f5-42d8-adfc-a1cc0673a2a7 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:31.962988 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d380a4-33f5-42d8-adfc-a1cc0673a2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:31.963260 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d380a4-33f5-42d8-adfc-a1cc0673a2a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:31.963974 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d380a4-33f5-42d8-adfc-a1cc0673a2a7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:31.964276 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d380a4-33f5-42d8-adfc-a1cc0673a2a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:31.965782 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d380a4-33f5-42d8-adfc-a1cc0673a2a7 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:31.980918 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 74/155] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Aug 5 14:06:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:31.986571 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d380a4-33f5-42d8-adfc-a1cc0673a2a7 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:32.016140 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:32.128734 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f1bd7-c382-4d97-a207-a4fb40cc98e3 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:32.187967 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d380a4-33f5-42d8-adfc-a1cc0673a2a7 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:32.197277 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaedd2e1-44f2-4bce-b62f-4cf6b6779a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:32.197989 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaedd2e1-44f2-4bce-b62f-4cf6b6779a66 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:32.198268 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaedd2e1-44f2-4bce-b62f-4cf6b6779a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:32.198537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaedd2e1-44f2-4bce-b62f-4cf6b6779a66 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:32.219088 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaedd2e1-44f2-4bce-b62f-4cf6b6779a66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:32.225064 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-aaedd2e1-44f2-4bce-b62f-4cf6b6779a66 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 14:06:32.225064 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:32.225064 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:32.225064 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:32.225064 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:32.225064 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:32.225064 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:32.225064 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:32.225064 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:32.225064 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:32.225064 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:32.225064 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:32.225064 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:32.225064 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:32.225064 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:32.225064 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:32.225999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:32.225999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:32.225999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:32.225999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:32.225999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:32.225999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:32.225999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:32.225999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:32.225999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:32.225999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:32.225999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:32.225999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:32.225999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:32.225999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:32.225999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:32.225999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:32.226949 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:32.226949 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:32.226949 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:32.226949 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:32.226949 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:32.226949 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:32.226949 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:32.226949 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:32.226949 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:32.226949 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:32.226949 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 14:06:32.226949 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:06:32.232993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 82/156] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Aug 5 14:06:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:06:32.238809 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1faa7-d3cb-4235-8d91-0aab6404099a None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:32.239094 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1faa7-d3cb-4235-8d91-0aab6404099a None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:32.239369 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1faa7-d3cb-4235-8d91-0aab6404099a None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:32.241309 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 83/157] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:32.244132 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8462172-5577-4230-9519-94e88bc407e8 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:32.244407 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8462172-5577-4230-9519-94e88bc407e8 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:32.244698 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8462172-5577-4230-9519-94e88bc407e8 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:32.248950 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 84/158] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:32.251041 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8462172-5577-4230-9519-94e88bc407e8 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:32.273139 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3537252-821e-4c56-8805-65f78489c0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:32.274182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3537252-821e-4c56-8805-65f78489c0dc None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:32.274681 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3537252-821e-4c56-8805-65f78489c0dc None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:32.275163 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3537252-821e-4c56-8805-65f78489c0dc None admin] PATH_INFO: `/v3/users/f62da411608041ce960346ac7faf6bbd` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:32.276709 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3537252-821e-4c56-8805-65f78489c0dc None admin] RBAC: Authorizing `identity:get_user(user_id=f62da411608041ce960346ac7faf6bbd)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:32.278371 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3537252-821e-4c56-8805-65f78489c0dc None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:32.280199 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 75/159] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:06:31 2022] GET /identity/v3/users/f62da411608041ce960346ac7faf6bbd => generated 289 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:06:32.281624 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eaefc3-43b0-48c0-95e0-9713a658fc42 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:32.282135 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eaefc3-43b0-48c0-95e0-9713a658fc42 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:32.282620 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eaefc3-43b0-48c0-95e0-9713a658fc42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:32.320816 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:32.321627 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-58eaefc3-43b0-48c0-95e0-9713a658fc42 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:32.322968 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-58eaefc3-43b0-48c0-95e0-9713a658fc42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:32.370126 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8462172-5577-4230-9519-94e88bc407e8 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:32.409183 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf85ed0-8c3f-477c-abab-e6291998e7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:32.410385 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf85ed0-8c3f-477c-abab-e6291998e7d6 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:32.410666 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf85ed0-8c3f-477c-abab-e6291998e7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:32.410954 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf85ed0-8c3f-477c-abab-e6291998e7d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:32.411657 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf85ed0-8c3f-477c-abab-e6291998e7d6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:32.411956 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf85ed0-8c3f-477c-abab-e6291998e7d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:32.414231 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf85ed0-8c3f-477c-abab-e6291998e7d6 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:32.427345 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 85/160] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:06:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:32.434035 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f535d3-0799-49ef-b54f-c56ff9e51244 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:32.434959 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f535d3-0799-49ef-b54f-c56ff9e51244 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:32.435242 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f535d3-0799-49ef-b54f-c56ff9e51244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:32.473126 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-a4f535d3-0799-49ef-b54f-c56ff9e51244 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:32.476804 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 76/161] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:32.477550 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eaefc3-43b0-48c0-95e0-9713a658fc42 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:32.478127 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-a4f535d3-0799-49ef-b54f-c56ff9e51244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:32.499319 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:32.596615 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 86/162] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:32.596615 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f535d3-0799-49ef-b54f-c56ff9e51244 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:32.679140 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:32.698206 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eaefc3-43b0-48c0-95e0-9713a658fc42 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:32.780210 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1654b03-1c4f-413a-abaa-13751b4f47e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:32.780808 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1654b03-1c4f-413a-abaa-13751b4f47e2 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:32.781084 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1654b03-1c4f-413a-abaa-13751b4f47e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:32.783001 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1654b03-1c4f-413a-abaa-13751b4f47e2 None admin] PATH_INFO: `/v3/projects/580a155c67b742b9bc398f50556f26cf` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:32.785473 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1654b03-1c4f-413a-abaa-13751b4f47e2 None admin] RBAC: Authorizing `identity:get_project(project_id=580a155c67b742b9bc398f50556f26cf)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:32.786912 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1654b03-1c4f-413a-abaa-13751b4f47e2 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:32.789146 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 77/163] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:06:32 2022] GET /identity/v3/projects/580a155c67b742b9bc398f50556f26cf => generated 298 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:32.789910 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1654b03-1c4f-413a-abaa-13751b4f47e2 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:32.793911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f535d3-0799-49ef-b54f-c56ff9e51244 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:32.840313 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2945b98-3cf5-47b1-a599-beec23520a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:32.840982 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2945b98-3cf5-47b1-a599-beec23520a47 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:32.841253 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2945b98-3cf5-47b1-a599-beec23520a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:32.841520 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2945b98-3cf5-47b1-a599-beec23520a47 None admin] PATH_INFO: `/v3/users/da4b0c24aa284fd59788ab74c424af63` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:32.844924 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2945b98-3cf5-47b1-a599-beec23520a47 None admin] RBAC: Authorizing `identity:get_user(user_id=da4b0c24aa284fd59788ab74c424af63)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:32.846360 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2945b98-3cf5-47b1-a599-beec23520a47 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:32.847955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 87/164] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:06:32 2022] GET /identity/v3/users/da4b0c24aa284fd59788ab74c424af63 => generated 275 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:32.848726 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2945b98-3cf5-47b1-a599-beec23520a47 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:32.958169 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:33.005142 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1654b03-1c4f-413a-abaa-13751b4f47e2 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:33.030171 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2945b98-3cf5-47b1-a599-beec23520a47 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:33.046568 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:33.071949 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3e8e0-9497-488c-8a07-0d92238511a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:33.072648 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3e8e0-9497-488c-8a07-0d92238511a5 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:33.072933 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3e8e0-9497-488c-8a07-0d92238511a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:33.073211 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3e8e0-9497-488c-8a07-0d92238511a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:33.077232 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dbf1a0-be83-490e-a337-f4203ced7b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:33.077907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dbf1a0-be83-490e-a337-f4203ced7b52 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:33.078184 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dbf1a0-be83-490e-a337-f4203ced7b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:33.078447 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dbf1a0-be83-490e-a337-f4203ced7b52 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:33.098767 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a3e8e0-9497-488c-8a07-0d92238511a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:33.104792 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-46a3e8e0-9497-488c-8a07-0d92238511a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:06:33.104792 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:33.104792 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:33.104792 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:33.104792 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:33.104792 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:33.104792 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:33.104792 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:33.104792 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:33.104792 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:33.104792 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:33.104792 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:33.104792 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:33.104792 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:33.104792 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:33.104792 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:33.105747 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:33.105747 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:33.105747 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:33.105747 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:33.105747 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:33.105747 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:33.105747 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:33.105747 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:33.105747 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:33.105747 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:33.105747 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:33.105747 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:33.105747 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:33.105747 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:33.105747 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:33.105747 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:33.106962 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:33.106962 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:33.106962 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:33.106962 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:33.106962 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:33.106962 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:33.106962 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:33.106962 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:33.106962 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:33.106962 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:33.106962 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:06:33.106962 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:06:33.106962 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 78/165] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Aug 5 14:06:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:06:33.106962 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3e8e0-9497-488c-8a07-0d92238511a5 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:33.108051 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dbf1a0-be83-490e-a337-f4203ced7b52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:33.113028 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-a9dbf1a0-be83-490e-a337-f4203ced7b52 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 14:06:33.113028 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:33.113028 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:33.113028 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:33.113028 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:33.113028 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:33.113028 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:33.113028 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:33.113028 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:33.113028 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:33.113028 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:33.113028 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:33.113028 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:33.113028 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:33.113028 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:33.113028 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:33.113970 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:33.113970 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:33.113970 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:33.113970 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:33.113970 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:33.113970 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:33.113970 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:33.113970 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:33.113970 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:33.113970 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:33.113970 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:33.113970 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:33.113970 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:33.113970 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:33.113970 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:33.113970 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:33.116392 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:33.116392 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:33.116392 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:33.116392 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:33.116392 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:33.116392 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:33.116392 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:33.116392 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:33.116392 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:33.116392 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:33.116392 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 14:06:33.116392 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:06:33.116392 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 88/166] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:06:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:06:33.116392 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dbf1a0-be83-490e-a337-f4203ced7b52 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:33.243246 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dbf1a0-be83-490e-a337-f4203ced7b52 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:33.271369 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3e8e0-9497-488c-8a07-0d92238511a5 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:33.278972 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4fc47e-689e-46b9-9910-3213dda9b76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:33.279608 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4fc47e-689e-46b9-9910-3213dda9b76f None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:33.279894 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4fc47e-689e-46b9-9910-3213dda9b76f None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:33.280278 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4fc47e-689e-46b9-9910-3213dda9b76f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:33.336028 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:33.337992 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee979d-64e6-43eb-bd2b-2a7fda0fc6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:33.338824 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee979d-64e6-43eb-bd2b-2a7fda0fc6f9 None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:33.338949 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee979d-64e6-43eb-bd2b-2a7fda0fc6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:33.339193 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee979d-64e6-43eb-bd2b-2a7fda0fc6f9 None admin] PATH_INFO: `/v3/projects/580a155c67b742b9bc398f50556f26cf/users/f62da411608041ce960346ac7faf6bbd/roles/8e4d742669e54e288bac0e45718908b8` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:33.344659 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ee979d-64e6-43eb-bd2b-2a7fda0fc6f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=580a155c67b742b9bc398f50556f26cf, user_id=f62da411608041ce960346ac7faf6bbd, role_id=8e4d742669e54e288bac0e45718908b8)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:33.346291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ee979d-64e6-43eb-bd2b-2a7fda0fc6f9 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:33.353018 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4fc47e-689e-46b9-9910-3213dda9b76f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:33.354505 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4fc47e-689e-46b9-9910-3213dda9b76f None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:33.434330 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:33.442705 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 79/167] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Aug 5 14:06:33 2022] PUT /identity/v3/projects/580a155c67b742b9bc398f50556f26cf/users/f62da411608041ce960346ac7faf6bbd/roles/8e4d742669e54e288bac0e45718908b8 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:06:33.443564 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee979d-64e6-43eb-bd2b-2a7fda0fc6f9 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:33.458306 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 89/168] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:06:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 14:06:33.459185 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4fc47e-689e-46b9-9910-3213dda9b76f None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:33.533491 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:33.566100 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4fc47e-689e-46b9-9910-3213dda9b76f None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:33.606037 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11ff9e5-d44b-44b8-baf4-3d05dd70223f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:33.606601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11ff9e5-d44b-44b8-baf4-3d05dd70223f None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:33.606878 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11ff9e5-d44b-44b8-baf4-3d05dd70223f None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:33.607394 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11ff9e5-d44b-44b8-baf4-3d05dd70223f None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:33.608232 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11ff9e5-d44b-44b8-baf4-3d05dd70223f None admin] RBAC: Adding query filter params (name=member) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:33.608688 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11ff9e5-d44b-44b8-baf4-3d05dd70223f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:33.610253 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11ff9e5-d44b-44b8-baf4-3d05dd70223f None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:33.620902 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 90/169] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Aug 5 14:06:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:33.621945 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11ff9e5-d44b-44b8-baf4-3d05dd70223f None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:33.648988 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee979d-64e6-43eb-bd2b-2a7fda0fc6f9 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:33.701375 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27869ef6-0b5a-408f-8672-f73e56e00618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:33.702005 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27869ef6-0b5a-408f-8672-f73e56e00618 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:33.702281 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27869ef6-0b5a-408f-8672-f73e56e00618 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:33.702552 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27869ef6-0b5a-408f-8672-f73e56e00618 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:33.703272 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27869ef6-0b5a-408f-8672-f73e56e00618 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:33.703571 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27869ef6-0b5a-408f-8672-f73e56e00618 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:33.704975 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27869ef6-0b5a-408f-8672-f73e56e00618 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:33.715098 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 80/170] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:06:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:33.720327 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27869ef6-0b5a-408f-8672-f73e56e00618 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:33.748388 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11ff9e5-d44b-44b8-baf4-3d05dd70223f None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:33.783127 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd2d7df-4174-436f-b717-004b2f5efa05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:33.783789 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2d7df-4174-436f-b717-004b2f5efa05 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:33.784074 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2d7df-4174-436f-b717-004b2f5efa05 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:33.784354 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2d7df-4174-436f-b717-004b2f5efa05 None admin] PATH_INFO: `/v3/projects/580a155c67b742b9bc398f50556f26cf` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:33.785588 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd2d7df-4174-436f-b717-004b2f5efa05 None admin] RBAC: Authorizing `identity:get_project(project_id=580a155c67b742b9bc398f50556f26cf)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:33.787071 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd2d7df-4174-436f-b717-004b2f5efa05 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:33.788847 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 91/171] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:06:33 2022] GET /identity/v3/projects/580a155c67b742b9bc398f50556f26cf => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:33.789629 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd2d7df-4174-436f-b717-004b2f5efa05 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:33.790261 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:33.812915 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:33.868532 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27869ef6-0b5a-408f-8672-f73e56e00618 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:33.929741 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e59c121-50c8-49fa-9db9-b7ec9fe7ca67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:33.930778 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59c121-50c8-49fa-9db9-b7ec9fe7ca67 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:33.931306 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59c121-50c8-49fa-9db9-b7ec9fe7ca67 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:33.931808 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59c121-50c8-49fa-9db9-b7ec9fe7ca67 None admin] PATH_INFO: `/v3/users/da4b0c24aa284fd59788ab74c424af63` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:33.933277 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e59c121-50c8-49fa-9db9-b7ec9fe7ca67 None admin] RBAC: Authorizing `identity:get_user(user_id=da4b0c24aa284fd59788ab74c424af63)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:33.935033 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e59c121-50c8-49fa-9db9-b7ec9fe7ca67 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:33.936922 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 81/172] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:06:33 2022] GET /identity/v3/users/da4b0c24aa284fd59788ab74c424af63 => generated 275 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:33.937951 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e59c121-50c8-49fa-9db9-b7ec9fe7ca67 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:33.963123 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd2d7df-4174-436f-b717-004b2f5efa05 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:34.028792 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8095e27d-4ee8-4ae1-99bf-b06209c20c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:34.031086 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095e27d-4ee8-4ae1-99bf-b06209c20c71 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:34.032658 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095e27d-4ee8-4ae1-99bf-b06209c20c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:34.033157 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095e27d-4ee8-4ae1-99bf-b06209c20c71 None admin] PATH_INFO: `/v3/users/5e3fbbd9c18b4be39ebc01f40d1df40f` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:34.035523 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8095e27d-4ee8-4ae1-99bf-b06209c20c71 None admin] RBAC: Authorizing `identity:get_user(user_id=5e3fbbd9c18b4be39ebc01f40d1df40f)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:34.037952 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8095e27d-4ee8-4ae1-99bf-b06209c20c71 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:34.040000 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 92/173] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:06:33 2022] GET /identity/v3/users/5e3fbbd9c18b4be39ebc01f40d1df40f => generated 247 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:34.042796 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bc986-8e74-4af1-95c6-309a876977e7 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:34.044075 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bc986-8e74-4af1-95c6-309a876977e7 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:34.045321 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bc986-8e74-4af1-95c6-309a876977e7 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:34.046960 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 93/174] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:34.049152 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424bc986-8e74-4af1-95c6-309a876977e7 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:34.116290 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e59c121-50c8-49fa-9db9-b7ec9fe7ca67 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:34.166293 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:34.169431 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:34.185697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0465f596-4afc-44d4-99d8-f05124ebb103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:34.186666 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0465f596-4afc-44d4-99d8-f05124ebb103 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:34.187173 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0465f596-4afc-44d4-99d8-f05124ebb103 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:34.187649 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0465f596-4afc-44d4-99d8-f05124ebb103 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:34.188731 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0465f596-4afc-44d4-99d8-f05124ebb103 None admin] RBAC: Adding query filter params (role.id=199fd3093a824ba585b1135c8bf363be, scope.project.id=580a155c67b742b9bc398f50556f26cf, user.id=da4b0c24aa284fd59788ab74c424af63) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:34.189248 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0465f596-4afc-44d4-99d8-f05124ebb103 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:34.190947 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0465f596-4afc-44d4-99d8-f05124ebb103 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:34.204238 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 82/175] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:06:33 2022] GET /identity/v3/role_assignments?user.id=da4b0c24aa284fd59788ab74c424af63&scope.project.id=580a155c67b742b9bc398f50556f26cf&role.id=199fd3093a824ba585b1135c8bf363be => generated 608 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:34.205563 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0465f596-4afc-44d4-99d8-f05124ebb103 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:34.236173 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424bc986-8e74-4af1-95c6-309a876977e7 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:34.313476 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb2a37d-d517-4a67-bf63-82f75013dd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:34.314162 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb2a37d-d517-4a67-bf63-82f75013dd98 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:34.314447 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb2a37d-d517-4a67-bf63-82f75013dd98 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:34.314723 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb2a37d-d517-4a67-bf63-82f75013dd98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:34.374887 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0465f596-4afc-44d4-99d8-f05124ebb103 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:34.448516 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb2a37d-d517-4a67-bf63-82f75013dd98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:34.450634 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb2a37d-d517-4a67-bf63-82f75013dd98 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:34.451408 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2522e8-b42e-4288-abd2-fa1df6019370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:34.452196 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2522e8-b42e-4288-abd2-fa1df6019370 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:34.452709 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2522e8-b42e-4288-abd2-fa1df6019370 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:34.453160 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2522e8-b42e-4288-abd2-fa1df6019370 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:34.529463 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:34.540165 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2522e8-b42e-4288-abd2-fa1df6019370 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:34.541647 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:34.543373 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2522e8-b42e-4288-abd2-fa1df6019370 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:34.585065 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 94/176] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:06:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 14:06:34.590046 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347f808-f10b-4253-a1bd-23d8194d0c25 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:34.590196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347f808-f10b-4253-a1bd-23d8194d0c25 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:34.590617 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347f808-f10b-4253-a1bd-23d8194d0c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:34.610589 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-5347f808-f10b-4253-a1bd-23d8194d0c25 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:34.611786 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-5347f808-f10b-4253-a1bd-23d8194d0c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:34.664917 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 83/177] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:06:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 14:06:34.667018 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5504df-d304-432f-8f4e-5635cfd97be8 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:34.667455 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5504df-d304-432f-8f4e-5635cfd97be8 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:34.667949 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5504df-d304-432f-8f4e-5635cfd97be8 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:34.670087 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 84/178] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:34.671307 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5939259-18cc-4036-a4d9-400c680f0fa0 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:34.671589 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5939259-18cc-4036-a4d9-400c680f0fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:34.671868 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5939259-18cc-4036-a4d9-400c680f0fa0 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:34.673222 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 85/179] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:34.675305 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5939259-18cc-4036-a4d9-400c680f0fa0 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:34.684908 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 95/180] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:34.685590 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5347f808-f10b-4253-a1bd-23d8194d0c25 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:34.788950 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:34.796581 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:34.799691 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5939259-18cc-4036-a4d9-400c680f0fa0 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:34.800686 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5347f808-f10b-4253-a1bd-23d8194d0c25 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:34.837470 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8151bd9c-a2e3-4379-9e14-b5a9d287e508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:34.838203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151bd9c-a2e3-4379-9e14-b5a9d287e508 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:34.838487 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151bd9c-a2e3-4379-9e14-b5a9d287e508 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:34.838757 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151bd9c-a2e3-4379-9e14-b5a9d287e508 None admin] PATH_INFO: `/v3/projects/f3df77a838364810aa3aa2a2a5bbf04e` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:34.838854 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f6fd1e-56c7-4a8f-8f16-daa940778d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:34.839538 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6fd1e-56c7-4a8f-8f16-daa940778d1b None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:34.839820 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6fd1e-56c7-4a8f-8f16-daa940778d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:34.839919 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8151bd9c-a2e3-4379-9e14-b5a9d287e508 None admin] RBAC: Authorizing `identity:get_project(project_id=f3df77a838364810aa3aa2a2a5bbf04e)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:34.840140 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6fd1e-56c7-4a8f-8f16-daa940778d1b None admin] PATH_INFO: `/v3/projects/580a155c67b742b9bc398f50556f26cf` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:34.841373 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f6fd1e-56c7-4a8f-8f16-daa940778d1b None admin] RBAC: Authorizing `identity:get_project(project_id=580a155c67b742b9bc398f50556f26cf)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:34.841484 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8151bd9c-a2e3-4379-9e14-b5a9d287e508 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:34.842833 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f6fd1e-56c7-4a8f-8f16-daa940778d1b None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:34.848034 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 86/181] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:06:34 2022] GET /identity/v3/projects/f3df77a838364810aa3aa2a2a5bbf04e => generated 312 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:34.848034 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adbe65c-7821-4b43-825e-5b1906df0493 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:34.848226 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 96/182] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:06:34 2022] GET /identity/v3/projects/580a155c67b742b9bc398f50556f26cf => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:34.848315 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adbe65c-7821-4b43-825e-5b1906df0493 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:34.848315 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adbe65c-7821-4b43-825e-5b1906df0493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:34.848466 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2de49f-cced-45d2-8cd8-80b08c95166c None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:34.848466 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2de49f-cced-45d2-8cd8-80b08c95166c None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:34.848466 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2de49f-cced-45d2-8cd8-80b08c95166c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:34.864347 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-8adbe65c-7821-4b43-825e-5b1906df0493 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:34.865602 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8adbe65c-7821-4b43-825e-5b1906df0493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:34.866293 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-fd2de49f-cced-45d2-8cd8-80b08c95166c None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:34.867652 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-fd2de49f-cced-45d2-8cd8-80b08c95166c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:34.938221 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 97/183] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:34.939094 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2de49f-cced-45d2-8cd8-80b08c95166c None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:34.956906 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 87/184] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:34.958264 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adbe65c-7821-4b43-825e-5b1906df0493 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:35.041074 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:35.047350 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2de49f-cced-45d2-8cd8-80b08c95166c None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:35.057412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:35.066729 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adbe65c-7821-4b43-825e-5b1906df0493 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:35.082974 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6b0d3-f34b-4f71-aedd-37e5fa2fd688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:35.083616 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6b0d3-f34b-4f71-aedd-37e5fa2fd688 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:35.083995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6b0d3-f34b-4f71-aedd-37e5fa2fd688 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:35.084196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6b0d3-f34b-4f71-aedd-37e5fa2fd688 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:35.096139 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed6b0d3-f34b-4f71-aedd-37e5fa2fd688 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:35.102287 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-7ed6b0d3-f34b-4f71-aedd-37e5fa2fd688 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:06:35.102287 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:35.102287 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:35.102287 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:35.102287 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:35.102287 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:35.102287 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:35.102287 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:35.102287 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:35.102287 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:35.102287 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:35.102287 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:35.102287 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:35.102287 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:35.102287 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:35.102287 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:35.103191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:35.103191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:35.103191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:35.103191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:35.103191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:35.103191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:35.103191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:35.103191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:35.103191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:35.103191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:35.103191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:35.103191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:35.103191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:35.103191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:35.103191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:35.103191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:35.109245 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:35.109245 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:35.109245 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:35.109245 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:35.109245 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:35.109245 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:35.109245 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:35.109245 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:35.109245 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:35.109245 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:35.109245 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:06:35.109245 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:06:35.109245 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 98/185] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Aug 5 14:06:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:06:35.109245 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6b0d3-f34b-4f71-aedd-37e5fa2fd688 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:35.109996 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01948026-d472-4214-8fab-868ca9768553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:35.109996 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01948026-d472-4214-8fab-868ca9768553 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:35.110442 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01948026-d472-4214-8fab-868ca9768553 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:35.110442 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01948026-d472-4214-8fab-868ca9768553 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:35.111250 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01948026-d472-4214-8fab-868ca9768553 None admin] RBAC: Adding query filter params (role.id=29967df108484a30a4c006cde2ae2940, scope.project.id=f3df77a838364810aa3aa2a2a5bbf04e, user.id=da4b0c24aa284fd59788ab74c424af63) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:35.111556 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01948026-d472-4214-8fab-868ca9768553 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:35.113033 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01948026-d472-4214-8fab-868ca9768553 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:35.119867 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 88/186] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:06:34 2022] GET /identity/v3/role_assignments?user.id=da4b0c24aa284fd59788ab74c424af63&scope.project.id=f3df77a838364810aa3aa2a2a5bbf04e&role.id=29967df108484a30a4c006cde2ae2940 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:35.120899 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01948026-d472-4214-8fab-868ca9768553 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:35.248548 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01948026-d472-4214-8fab-868ca9768553 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:35.258694 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6b0d3-f34b-4f71-aedd-37e5fa2fd688 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:35.288927 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77fa1a1-66d1-405f-9170-45bd4131d83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:35.289702 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77fa1a1-66d1-405f-9170-45bd4131d83d None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:35.289983 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77fa1a1-66d1-405f-9170-45bd4131d83d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:35.290257 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77fa1a1-66d1-405f-9170-45bd4131d83d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:35.291493 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77fa1a1-66d1-405f-9170-45bd4131d83d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:35.292948 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77fa1a1-66d1-405f-9170-45bd4131d83d None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:35.295183 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 89/187] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:06:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:35.296017 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77fa1a1-66d1-405f-9170-45bd4131d83d None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:35.305648 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7f784-8a04-4a7c-8118-37f5bf5f72c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:35.307134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7f784-8a04-4a7c-8118-37f5bf5f72c5 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:35.307427 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7f784-8a04-4a7c-8118-37f5bf5f72c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:35.307708 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7f784-8a04-4a7c-8118-37f5bf5f72c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:35.309268 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a7f784-8a04-4a7c-8118-37f5bf5f72c5 None admin] RBAC: Adding query filter params (role.id=299c794c7acb4a5ca36288c3191aad10, scope.project.id=580a155c67b742b9bc398f50556f26cf, user.id=5e3fbbd9c18b4be39ebc01f40d1df40f) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:35.309589 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a7f784-8a04-4a7c-8118-37f5bf5f72c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:35.311052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a7f784-8a04-4a7c-8118-37f5bf5f72c5 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:35.317660 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 99/188] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:06:35 2022] GET /identity/v3/role_assignments?user.id=5e3fbbd9c18b4be39ebc01f40d1df40f&scope.project.id=580a155c67b742b9bc398f50556f26cf&role.id=299c794c7acb4a5ca36288c3191aad10 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:35.318569 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7f784-8a04-4a7c-8118-37f5bf5f72c5 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:35.346977 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:35.356595 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:35.460076 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7f784-8a04-4a7c-8118-37f5bf5f72c5 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:35.474688 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77fa1a1-66d1-405f-9170-45bd4131d83d None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:35.508343 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0568f7-6134-4cd7-97e8-b06c08101b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:35.509280 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0568f7-6134-4cd7-97e8-b06c08101b96 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:35.509665 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0568f7-6134-4cd7-97e8-b06c08101b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:35.510037 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0568f7-6134-4cd7-97e8-b06c08101b96 None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:35.510874 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0568f7-6134-4cd7-97e8-b06c08101b96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:35.511284 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0568f7-6134-4cd7-97e8-b06c08101b96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:35.512958 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0568f7-6134-4cd7-97e8-b06c08101b96 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:35.523510 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 100/189] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:06:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:35.524626 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0568f7-6134-4cd7-97e8-b06c08101b96 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:35.527297 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5775f-38a5-4340-808e-7a3619ae9a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:35.527956 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5775f-38a5-4340-808e-7a3619ae9a37 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:35.528234 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5775f-38a5-4340-808e-7a3619ae9a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:35.529774 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5775f-38a5-4340-808e-7a3619ae9a37 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:35.541744 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c5775f-38a5-4340-808e-7a3619ae9a37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:35.546730 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-e0c5775f-38a5-4340-808e-7a3619ae9a37 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 14:06:35.546730 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:35.546730 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:35.546730 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:35.546730 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:35.546730 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:35.546730 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:35.546730 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:35.546730 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:35.546730 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:35.546730 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:35.546730 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:35.546730 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:35.546730 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:35.546730 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:35.546730 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:35.547617 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:35.547617 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:35.547617 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:35.547617 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:35.547617 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:35.547617 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:35.547617 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:35.547617 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:35.547617 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:35.547617 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:35.547617 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:35.547617 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:35.547617 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:35.547617 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:35.547617 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:35.547617 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:35.548447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:35.548447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:35.548447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:35.548447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:35.548447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:35.548447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:35.548447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:35.548447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:35.548447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:35.548447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:35.548447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 14:06:35.548447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:06:35.548447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 90/190] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Aug 5 14:06:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:06:35.550073 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5775f-38a5-4340-808e-7a3619ae9a37 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:35.638089 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:35.653189 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:35.659366 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0568f7-6134-4cd7-97e8-b06c08101b96 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:35.671669 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5775f-38a5-4340-808e-7a3619ae9a37 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:35.697464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ac3cb-3eeb-477f-91af-b5c60498e4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:35.698340 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ac3cb-3eeb-477f-91af-b5c60498e4cb None admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:35.698755 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ac3cb-3eeb-477f-91af-b5c60498e4cb None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:35.699196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ac3cb-3eeb-477f-91af-b5c60498e4cb None admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:35.700198 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6ac3cb-3eeb-477f-91af-b5c60498e4cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:35.702876 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6ac3cb-3eeb-477f-91af-b5c60498e4cb None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:35.706236 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11439e3-3471-4d77-a8fb-ecd0bed70d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:35.706873 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11439e3-3471-4d77-a8fb-ecd0bed70d53 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:35.707145 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11439e3-3471-4d77-a8fb-ecd0bed70d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:35.707416 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11439e3-3471-4d77-a8fb-ecd0bed70d53 None admin] PATH_INFO: `/v3/users/ffbe094b250840fa86b8b1ecf2ce214d` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:35.720949 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11439e3-3471-4d77-a8fb-ecd0bed70d53 None admin] RBAC: Authorizing `identity:get_user(user_id=ffbe094b250840fa86b8b1ecf2ce214d)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:35.722374 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11439e3-3471-4d77-a8fb-ecd0bed70d53 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:35.724031 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 91/191] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:06:35 2022] GET /identity/v3/users/ffbe094b250840fa86b8b1ecf2ce214d => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:35.728158 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11439e3-3471-4d77-a8fb-ecd0bed70d53 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:35.799021 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 101/192] 10.222.0.26 () {60 vars in 1132 bytes} [Fri Aug 5 14:06:35 2022] POST /identity/v3/users => generated 297 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:06:35.799706 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ac3cb-3eeb-477f-91af-b5c60498e4cb None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:35.834263 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11439e3-3471-4d77-a8fb-ecd0bed70d53 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:35.884392 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc47712b-2de5-4645-beef-97f10282e7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:35.885065 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47712b-2de5-4645-beef-97f10282e7a6 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:35.885345 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47712b-2de5-4645-beef-97f10282e7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:35.885623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47712b-2de5-4645-beef-97f10282e7a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:35.886374 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc47712b-2de5-4645-beef-97f10282e7a6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:35.886683 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc47712b-2de5-4645-beef-97f10282e7a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:35.888087 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc47712b-2de5-4645-beef-97f10282e7a6 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:35.897182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 92/193] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:06:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:35.897182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b4512-d478-43d2-9c09-15d81d1bfebc None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:35.897182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b4512-d478-43d2-9c09-15d81d1bfebc None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:35.897182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b4512-d478-43d2-9c09-15d81d1bfebc None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:35.897182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 93/194] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:35.902768 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b4512-d478-43d2-9c09-15d81d1bfebc None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:35.905886 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:35.941276 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ac3cb-3eeb-477f-91af-b5c60498e4cb None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:35.984442 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056edaa2-f5af-47ba-a33c-ef3337ec19f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:35.985081 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056edaa2-f5af-47ba-a33c-ef3337ec19f8 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:35.985371 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056edaa2-f5af-47ba-a33c-ef3337ec19f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:35.985638 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056edaa2-f5af-47ba-a33c-ef3337ec19f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:35.997881 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:36.026144 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b4512-d478-43d2-9c09-15d81d1bfebc None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:36.075473 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd605720-224c-4aad-9600-a6b9bf0ddd07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:36.076071 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd605720-224c-4aad-9600-a6b9bf0ddd07 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:36.076319 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd605720-224c-4aad-9600-a6b9bf0ddd07 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:36.076614 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd605720-224c-4aad-9600-a6b9bf0ddd07 None admin] PATH_INFO: `/v3/users/ffbe094b250840fa86b8b1ecf2ce214d` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:36.077875 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056edaa2-f5af-47ba-a33c-ef3337ec19f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:36.079340 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056edaa2-f5af-47ba-a33c-ef3337ec19f8 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:36.081285 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd605720-224c-4aad-9600-a6b9bf0ddd07 None admin] RBAC: Authorizing `identity:get_user(user_id=ffbe094b250840fa86b8b1ecf2ce214d)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:36.082917 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd605720-224c-4aad-9600-a6b9bf0ddd07 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:36.085211 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 94/195] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:06:35 2022] GET /identity/v3/users/ffbe094b250840fa86b8b1ecf2ce214d => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:36.086398 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb200b-b947-4184-a839-929ed1f33278 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:36.086655 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb200b-b947-4184-a839-929ed1f33278 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:36.086981 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb200b-b947-4184-a839-929ed1f33278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:36.105888 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-70cb200b-b947-4184-a839-929ed1f33278 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:36.107087 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-70cb200b-b947-4184-a839-929ed1f33278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:36.188135 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:36.193063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 95/196] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:36.194043 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb200b-b947-4184-a839-929ed1f33278 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:36.205309 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 102/197] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:06:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 14:06:36.212253 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056edaa2-f5af-47ba-a33c-ef3337ec19f8 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:36.267198 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:36.311800 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb200b-b947-4184-a839-929ed1f33278 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:36.348442 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be23ee80-446b-4550-988b-7da28a2c5081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:36.348993 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23ee80-446b-4550-988b-7da28a2c5081 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:36.349380 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23ee80-446b-4550-988b-7da28a2c5081 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:36.349661 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23ee80-446b-4550-988b-7da28a2c5081 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:36.352539 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056edaa2-f5af-47ba-a33c-ef3337ec19f8 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:36.407098 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee4f18e-e04b-480a-935f-e60b3385b42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:36.407812 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee4f18e-e04b-480a-935f-e60b3385b42b None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:36.408097 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee4f18e-e04b-480a-935f-e60b3385b42b None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:36.408368 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee4f18e-e04b-480a-935f-e60b3385b42b None admin] PATH_INFO: `/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:36.410510 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee4f18e-e04b-480a-935f-e60b3385b42b None admin] RBAC: Authorizing `identity:get_project(project_id=ecbbd8ae8eed4ce5b441221e8e17663b)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:36.413176 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee4f18e-e04b-480a-935f-e60b3385b42b None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:36.415422 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 103/198] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:06:36 2022] GET /identity/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:06:36.416869 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee4f18e-e04b-480a-935f-e60b3385b42b None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:36.430086 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:36.432813 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be23ee80-446b-4550-988b-7da28a2c5081 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:36.434456 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be23ee80-446b-4550-988b-7da28a2c5081 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:36.552564 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee4f18e-e04b-480a-935f-e60b3385b42b None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:36.560993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:36.587399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 96/199] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:06:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 14:06:36.587399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be23ee80-446b-4550-988b-7da28a2c5081 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:36.603740 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91652cc4-897b-41fa-a076-cfdfcd17f923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:36.604973 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91652cc4-897b-41fa-a076-cfdfcd17f923 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:36.605484 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91652cc4-897b-41fa-a076-cfdfcd17f923 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:36.606207 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91652cc4-897b-41fa-a076-cfdfcd17f923 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:36.621955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91652cc4-897b-41fa-a076-cfdfcd17f923 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:36.629049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-91652cc4-897b-41fa-a076-cfdfcd17f923 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 14:06:36.629049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:36.629049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:36.629049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:36.629049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:36.629049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:36.629049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:36.629049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:36.629049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:36.629049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:36.629049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:36.629049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:36.629049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:36.629049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:36.629049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:36.629049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:36.630153 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:36.630153 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:36.630153 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:36.630153 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:36.630153 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:36.630153 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:36.630153 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:36.630153 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:36.630153 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:36.630153 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:36.630153 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:36.630153 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:36.630153 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:36.630153 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:36.630153 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:36.630153 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:36.631806 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:36.631806 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:36.631806 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:36.631806 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:36.631806 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:36.631806 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:36.631806 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:36.631806 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:36.631806 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:36.631806 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:36.631806 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 14:06:36.631806 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:06:36.633985 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 104/200] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:06:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:06:36.635664 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91652cc4-897b-41fa-a076-cfdfcd17f923 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:36.704496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:36.709805 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be23ee80-446b-4550-988b-7da28a2c5081 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:36.758581 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c145f515-500b-489b-94a5-fc32b9513461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:36.759351 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145f515-500b-489b-94a5-fc32b9513461 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:36.759729 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145f515-500b-489b-94a5-fc32b9513461 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:36.760093 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145f515-500b-489b-94a5-fc32b9513461 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:36.760664 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91652cc4-897b-41fa-a076-cfdfcd17f923 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:36.761105 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c145f515-500b-489b-94a5-fc32b9513461 None admin] RBAC: Adding query filter params (role.id=299c794c7acb4a5ca36288c3191aad10, scope.project.id=ecbbd8ae8eed4ce5b441221e8e17663b, user.id=ffbe094b250840fa86b8b1ecf2ce214d) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:36.761493 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c145f515-500b-489b-94a5-fc32b9513461 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:36.763077 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c145f515-500b-489b-94a5-fc32b9513461 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:36.776443 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 97/201] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:06:36 2022] GET /identity/v3/role_assignments?user.id=ffbe094b250840fa86b8b1ecf2ce214d&scope.project.id=ecbbd8ae8eed4ce5b441221e8e17663b&role.id=299c794c7acb4a5ca36288c3191aad10 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:36.776443 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c145f515-500b-489b-94a5-fc32b9513461 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:36.834041 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b6d74-0133-43c2-9e1c-8da61c35b0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:36.836054 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b6d74-0133-43c2-9e1c-8da61c35b0a6 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:36.841168 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b6d74-0133-43c2-9e1c-8da61c35b0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:36.841168 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b6d74-0133-43c2-9e1c-8da61c35b0a6 None admin] PATH_INFO: `/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:36.847292 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9b6d74-0133-43c2-9e1c-8da61c35b0a6 None admin] RBAC: Authorizing `identity:get_project(project_id=ecbbd8ae8eed4ce5b441221e8e17663b)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:36.848873 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9b6d74-0133-43c2-9e1c-8da61c35b0a6 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:36.851479 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 105/202] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:06:36 2022] GET /identity/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:36.861470 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b6d74-0133-43c2-9e1c-8da61c35b0a6 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:36.930654 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c145f515-500b-489b-94a5-fc32b9513461 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:36.972734 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe5289-3a67-4e9b-b8da-31a2fb810c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:36.973716 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe5289-3a67-4e9b-b8da-31a2fb810c1b None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:36.974196 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe5289-3a67-4e9b-b8da-31a2fb810c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:36.974711 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:36.974848 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe5289-3a67-4e9b-b8da-31a2fb810c1b None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:36.977504 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe5289-3a67-4e9b-b8da-31a2fb810c1b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:36.978045 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe5289-3a67-4e9b-b8da-31a2fb810c1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:36.979712 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe5289-3a67-4e9b-b8da-31a2fb810c1b None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:36.993358 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 98/203] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Aug 5 14:06:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:36.999966 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe5289-3a67-4e9b-b8da-31a2fb810c1b None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:37.035659 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:37.082129 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b6d74-0133-43c2-9e1c-8da61c35b0a6 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:37.155723 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158895a9-e47a-4399-bd68-f440e87f5621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:37.156727 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158895a9-e47a-4399-bd68-f440e87f5621 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:37.157432 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158895a9-e47a-4399-bd68-f440e87f5621 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:37.158756 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158895a9-e47a-4399-bd68-f440e87f5621 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:37.161469 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158895a9-e47a-4399-bd68-f440e87f5621 None admin] RBAC: Adding query filter params (role.id=199fd3093a824ba585b1135c8bf363be, scope.project.id=ecbbd8ae8eed4ce5b441221e8e17663b, user.id=ffbe094b250840fa86b8b1ecf2ce214d) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:37.161995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158895a9-e47a-4399-bd68-f440e87f5621 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:37.164843 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158895a9-e47a-4399-bd68-f440e87f5621 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:37.180440 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 106/204] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:06:36 2022] GET /identity/v3/role_assignments?user.id=ffbe094b250840fa86b8b1ecf2ce214d&scope.project.id=ecbbd8ae8eed4ce5b441221e8e17663b&role.id=199fd3093a824ba585b1135c8bf363be => generated 259 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:37.194526 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe5289-3a67-4e9b-b8da-31a2fb810c1b None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:37.242678 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f6405-d4da-4fb7-8bf4-46c5771418dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:37.243798 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f6405-d4da-4fb7-8bf4-46c5771418dd None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:37.244304 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f6405-d4da-4fb7-8bf4-46c5771418dd None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:37.244832 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f6405-d4da-4fb7-8bf4-46c5771418dd None admin] PATH_INFO: `/v3/users/f62da411608041ce960346ac7faf6bbd` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:37.246354 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f6405-d4da-4fb7-8bf4-46c5771418dd None admin] RBAC: Authorizing `identity:get_user(user_id=f62da411608041ce960346ac7faf6bbd)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:37.248305 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f6405-d4da-4fb7-8bf4-46c5771418dd None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:37.250187 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 99/205] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:06:36 2022] GET /identity/v3/users/f62da411608041ce960346ac7faf6bbd => generated 289 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:37.257599 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158895a9-e47a-4399-bd68-f440e87f5621 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:37.409678 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:37.414093 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158895a9-e47a-4399-bd68-f440e87f5621 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:37.457667 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd31c8c7-3914-43d1-96b3-db7634bb3c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:37.458781 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd31c8c7-3914-43d1-96b3-db7634bb3c84 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:37.459256 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd31c8c7-3914-43d1-96b3-db7634bb3c84 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:37.460331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd31c8c7-3914-43d1-96b3-db7634bb3c84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:37.544760 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd31c8c7-3914-43d1-96b3-db7634bb3c84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:37.546201 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd31c8c7-3914-43d1-96b3-db7634bb3c84 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:37.661302 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:37.668115 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 107/206] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:06:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 14:06:37.673927 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f6405-d4da-4fb7-8bf4-46c5771418dd None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:37.806013 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:37.818709 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f6405-d4da-4fb7-8bf4-46c5771418dd None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:37.868640 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671de980-0049-4e9e-87da-fca69bff2086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:37.869627 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671de980-0049-4e9e-87da-fca69bff2086 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:37.870133 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671de980-0049-4e9e-87da-fca69bff2086 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:37.870605 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671de980-0049-4e9e-87da-fca69bff2086 None admin] PATH_INFO: `/v3/projects/580a155c67b742b9bc398f50556f26cf` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:37.871978 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671de980-0049-4e9e-87da-fca69bff2086 None admin] RBAC: Authorizing `identity:get_project(project_id=580a155c67b742b9bc398f50556f26cf)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:37.873894 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671de980-0049-4e9e-87da-fca69bff2086 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:37.875785 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 100/207] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:06:37 2022] GET /identity/v3/projects/580a155c67b742b9bc398f50556f26cf => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:37.882553 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd31c8c7-3914-43d1-96b3-db7634bb3c84 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:37.998216 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd31c8c7-3914-43d1-96b3-db7634bb3c84 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:38.040368 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c64e2-fac3-4211-b3d4-2c6550374817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:38.041015 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c64e2-fac3-4211-b3d4-2c6550374817 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:38.041289 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c64e2-fac3-4211-b3d4-2c6550374817 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:38.041665 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c64e2-fac3-4211-b3d4-2c6550374817 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:38.042466 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c64e2-fac3-4211-b3d4-2c6550374817 None admin] RBAC: Adding query filter params (role.id=8e4d742669e54e288bac0e45718908b8, scope.project.id=580a155c67b742b9bc398f50556f26cf, user.id=f62da411608041ce960346ac7faf6bbd) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:38.042768 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c64e2-fac3-4211-b3d4-2c6550374817 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:38.044206 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c64e2-fac3-4211-b3d4-2c6550374817 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:38.050876 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 108/208] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:06:37 2022] GET /identity/v3/role_assignments?user.id=f62da411608041ce960346ac7faf6bbd&scope.project.id=580a155c67b742b9bc398f50556f26cf&role.id=8e4d742669e54e288bac0e45718908b8 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:38.173655 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dfc6fd-2500-4c25-ace9-2d9d5be8a81d None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:38.174290 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dfc6fd-2500-4c25-ace9-2d9d5be8a81d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:38.174768 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dfc6fd-2500-4c25-ace9-2d9d5be8a81d None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:38.176737 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 101/209] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:38.182638 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8126a-e90b-46ca-8a5e-bf634ef159a7 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:38.182955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8126a-e90b-46ca-8a5e-bf634ef159a7 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:38.183187 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8126a-e90b-46ca-8a5e-bf634ef159a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:38.213572 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-16a8126a-e90b-46ca-8a5e-bf634ef159a7 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:38.214788 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-16a8126a-e90b-46ca-8a5e-bf634ef159a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:38.290921 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:38.310847 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 109/210] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:38.348340 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dfc6fd-2500-4c25-ace9-2d9d5be8a81d None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:38.355588 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af5c64-0af7-4a4e-83a2-da4e62d045ae None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:38.355588 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af5c64-0af7-4a4e-83a2-da4e62d045ae None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:38.355588 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af5c64-0af7-4a4e-83a2-da4e62d045ae None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:38.357105 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 110/211] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:38.362043 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ea85e-03aa-470b-9767-3173c8776f9b None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:38.362391 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ea85e-03aa-470b-9767-3173c8776f9b None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:38.362672 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ea85e-03aa-470b-9767-3173c8776f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:38.385486 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-121ea85e-03aa-470b-9767-3173c8776f9b None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:38.386671 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-121ea85e-03aa-470b-9767-3173c8776f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:38.473784 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 111/212] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 14:06:38.489291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dfc6fd-2500-4c25-ace9-2d9d5be8a81d None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:38.501910 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ea85e-03aa-470b-9767-3173c8776f9b None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:38.544731 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d68e7-9517-45ab-8169-e52a101159fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:38.545684 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d68e7-9517-45ab-8169-e52a101159fd None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:38.546188 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d68e7-9517-45ab-8169-e52a101159fd None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:38.546690 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d68e7-9517-45ab-8169-e52a101159fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:38.547930 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2d68e7-9517-45ab-8169-e52a101159fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:38.549487 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2d68e7-9517-45ab-8169-e52a101159fd None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:38.551347 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 102/213] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:06:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:38.557517 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d68e7-9517-45ab-8169-e52a101159fd None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:38.597535 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:38.656710 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ea85e-03aa-470b-9767-3173c8776f9b None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:38.682072 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:38.693337 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d68e7-9517-45ab-8169-e52a101159fd None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:38.694860 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e0492a-91e9-4ce1-b92d-660d99f3ee30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:38.695722 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0492a-91e9-4ce1-b92d-660d99f3ee30 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:38.696065 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0492a-91e9-4ce1-b92d-660d99f3ee30 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:38.696505 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0492a-91e9-4ce1-b92d-660d99f3ee30 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:38.707225 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e0492a-91e9-4ce1-b92d-660d99f3ee30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:38.713556 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-e1e0492a-91e9-4ce1-b92d-660d99f3ee30 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 14:06:38.713556 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:38.713556 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:38.713556 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:38.713556 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:38.713556 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:38.713556 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:38.713556 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:38.713556 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:38.713556 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:38.713556 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:38.713556 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:38.713556 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:38.713556 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:38.713556 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:38.713556 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:38.714450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:38.714450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:38.714450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:38.714450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:38.714450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:38.714450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:38.714450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:38.714450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:38.714450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:38.714450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:38.714450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:38.714450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:38.714450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:38.714450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:38.714450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:38.714450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:38.715402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:38.715402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:38.715402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:38.715402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:38.715402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:38.715402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:38.715402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:38.715402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:38.715402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:38.715402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:38.715402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 14:06:38.715402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:06:38.715402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 112/214] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:06:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:06:38.722926 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e0492a-91e9-4ce1-b92d-660d99f3ee30 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:38.732330 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5d7e3-09a8-43eb-b59a-3664d2da19a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:38.733105 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5d7e3-09a8-43eb-b59a-3664d2da19a2 None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:38.733389 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5d7e3-09a8-43eb-b59a-3664d2da19a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:38.733662 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5d7e3-09a8-43eb-b59a-3664d2da19a2 None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:38.734357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f5d7e3-09a8-43eb-b59a-3664d2da19a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:38.735980 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f5d7e3-09a8-43eb-b59a-3664d2da19a2 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:38.838567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 103/215] 10.222.0.26 () {60 vars in 1132 bytes} [Fri Aug 5 14:06:38 2022] POST /identity/v3/users => generated 297 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:06:38.841527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e0492a-91e9-4ce1-b92d-660d99f3ee30 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:38.886831 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd157209-9b37-44ea-82af-49ba029e1368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:38.888306 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd157209-9b37-44ea-82af-49ba029e1368 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:38.888841 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd157209-9b37-44ea-82af-49ba029e1368 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:38.889335 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd157209-9b37-44ea-82af-49ba029e1368 None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:38.890338 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd157209-9b37-44ea-82af-49ba029e1368 None admin] RBAC: Adding query filter params (name=member) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:38.890931 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd157209-9b37-44ea-82af-49ba029e1368 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:38.892691 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd157209-9b37-44ea-82af-49ba029e1368 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:38.908454 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 113/216] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Aug 5 14:06:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:38.913654 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5d7e3-09a8-43eb-b59a-3664d2da19a2 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:39.011170 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:39.026728 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5d7e3-09a8-43eb-b59a-3664d2da19a2 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:39.063177 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42546f9d-db0a-4492-acaa-f68dde3968d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:39.063819 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42546f9d-db0a-4492-acaa-f68dde3968d3 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:39.064093 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42546f9d-db0a-4492-acaa-f68dde3968d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:39.064362 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42546f9d-db0a-4492-acaa-f68dde3968d3 None admin] PATH_INFO: `/v3/users/182e7de0d85f40f0941829018ce80365` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:39.079507 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42546f9d-db0a-4492-acaa-f68dde3968d3 None admin] RBAC: Authorizing `identity:get_user(user_id=182e7de0d85f40f0941829018ce80365)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:39.080983 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42546f9d-db0a-4492-acaa-f68dde3968d3 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:39.082969 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 104/217] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:06:38 2022] GET /identity/v3/users/182e7de0d85f40f0941829018ce80365 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:39.089086 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd157209-9b37-44ea-82af-49ba029e1368 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:39.142916 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:39.201707 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd157209-9b37-44ea-82af-49ba029e1368 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:39.239418 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48061bbd-75bc-454a-aa15-bb31d953bb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:39.240192 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48061bbd-75bc-454a-aa15-bb31d953bb30 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:39.241121 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48061bbd-75bc-454a-aa15-bb31d953bb30 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:39.241485 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48061bbd-75bc-454a-aa15-bb31d953bb30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:39.269284 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfded7-5929-4781-9d39-7aa83ed63fb8 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:39.270108 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfded7-5929-4781-9d39-7aa83ed63fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:39.270551 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfded7-5929-4781-9d39-7aa83ed63fb8 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:39.273624 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 105/218] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:06:39.278870 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c299e-7b72-4370-9e5d-6aea27ff5ee3 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:39.279253 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c299e-7b72-4370-9e5d-6aea27ff5ee3 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:39.279621 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c299e-7b72-4370-9e5d-6aea27ff5ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:39.311793 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-186c299e-7b72-4370-9e5d-6aea27ff5ee3 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:39.313030 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-186c299e-7b72-4370-9e5d-6aea27ff5ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:39.331350 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48061bbd-75bc-454a-aa15-bb31d953bb30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:39.333683 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48061bbd-75bc-454a-aa15-bb31d953bb30 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:39.388706 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 106/219] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:39.395288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:39.408038 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c299e-7b72-4370-9e5d-6aea27ff5ee3 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:39.461661 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 114/220] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:06:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 14:06:39.467463 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48061bbd-75bc-454a-aa15-bb31d953bb30 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:39.504240 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:39.547460 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c299e-7b72-4370-9e5d-6aea27ff5ee3 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:39.589662 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f53ef4c-3472-4c92-980b-685c03eea169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:39.590299 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f53ef4c-3472-4c92-980b-685c03eea169 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:39.590575 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f53ef4c-3472-4c92-980b-685c03eea169 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:39.590847 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f53ef4c-3472-4c92-980b-685c03eea169 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:39.603017 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f53ef4c-3472-4c92-980b-685c03eea169 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:39.609021 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-5f53ef4c-3472-4c92-980b-685c03eea169 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:06:39.609021 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:39.609021 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:39.609021 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:39.609021 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:39.609021 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:39.609021 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:39.609021 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:39.609021 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:39.609021 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:39.609021 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:39.609021 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:39.609021 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:39.609021 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:39.609021 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:39.609021 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:39.611361 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:39.611361 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:39.611361 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:39.611361 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:39.611361 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:39.611361 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:39.611361 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:39.611361 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:39.611361 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:39.611361 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:39.611361 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:39.611361 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:39.611361 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:39.611361 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:39.611361 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:39.611361 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:39.612290 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:39.612290 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:39.612290 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:39.612290 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:39.612290 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:39.612290 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:39.612290 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:39.612290 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:39.612290 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:39.612290 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:39.612290 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:06:39.612290 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:06:39.612290 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 107/221] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Aug 5 14:06:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:06:39.617177 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f53ef4c-3472-4c92-980b-685c03eea169 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:39.669563 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48061bbd-75bc-454a-aa15-bb31d953bb30 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:39.707444 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0858b-3d12-48d6-b362-dc062ebce856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:39.708315 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0858b-3d12-48d6-b362-dc062ebce856 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:39.708536 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0858b-3d12-48d6-b362-dc062ebce856 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:39.708811 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0858b-3d12-48d6-b362-dc062ebce856 None admin] PATH_INFO: `/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:39.710138 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c0858b-3d12-48d6-b362-dc062ebce856 None admin] RBAC: Authorizing `identity:get_project(project_id=ecbbd8ae8eed4ce5b441221e8e17663b)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:39.711780 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c0858b-3d12-48d6-b362-dc062ebce856 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:39.718542 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 115/222] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:06:39 2022] GET /identity/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b => generated 302 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:39.720038 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c1d38-c4ce-42f5-b850-09a988c2efda None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:39.720347 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c1d38-c4ce-42f5-b850-09a988c2efda None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:39.720708 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c1d38-c4ce-42f5-b850-09a988c2efda None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:39.722016 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 116/223] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:39.727708 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17c1d38-c4ce-42f5-b850-09a988c2efda None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:39.741302 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:39.743259 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f53ef4c-3472-4c92-980b-685c03eea169 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:39.789276 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a978c8-a9cb-4aa6-b292-c3c8733ac6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:39.789901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a978c8-a9cb-4aa6-b292-c3c8733ac6e4 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:39.790180 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a978c8-a9cb-4aa6-b292-c3c8733ac6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:39.790450 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a978c8-a9cb-4aa6-b292-c3c8733ac6e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:39.791170 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a978c8-a9cb-4aa6-b292-c3c8733ac6e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:39.791472 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a978c8-a9cb-4aa6-b292-c3c8733ac6e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:39.793416 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a978c8-a9cb-4aa6-b292-c3c8733ac6e4 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:39.795233 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:39.806327 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 108/224] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:06:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:39.809653 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e2cbc-d0ae-4f13-be5c-57a4674996d3 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:39.810174 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e2cbc-d0ae-4f13-be5c-57a4674996d3 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:39.810654 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e2cbc-d0ae-4f13-be5c-57a4674996d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:39.836568 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-c93e2cbc-d0ae-4f13-be5c-57a4674996d3 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:39.837781 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c93e2cbc-d0ae-4f13-be5c-57a4674996d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:39.867786 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17c1d38-c4ce-42f5-b850-09a988c2efda None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:39.912146 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 109/225] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:39.920641 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e2cbc-d0ae-4f13-be5c-57a4674996d3 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:39.920758 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a28a86-61a6-4844-a9fe-5786905253c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:39.920758 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a28a86-61a6-4844-a9fe-5786905253c6 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:39.920758 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a28a86-61a6-4844-a9fe-5786905253c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:39.920758 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a28a86-61a6-4844-a9fe-5786905253c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:39.921311 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a28a86-61a6-4844-a9fe-5786905253c6 None admin] RBAC: Adding query filter params (role.id=29967df108484a30a4c006cde2ae2940, scope.project.id=ecbbd8ae8eed4ce5b441221e8e17663b, user.id=182e7de0d85f40f0941829018ce80365) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:39.921311 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a28a86-61a6-4844-a9fe-5786905253c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:39.927181 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a28a86-61a6-4844-a9fe-5786905253c6 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:39.937022 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 117/226] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:06:39 2022] GET /identity/v3/role_assignments?user.id=182e7de0d85f40f0941829018ce80365&scope.project.id=ecbbd8ae8eed4ce5b441221e8e17663b&role.id=29967df108484a30a4c006cde2ae2940 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:06:39.942645 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a28a86-61a6-4844-a9fe-5786905253c6 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:40.044192 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:40.071984 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a28a86-61a6-4844-a9fe-5786905253c6 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:40.114334 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17537713-092a-4e75-b0e3-227aa35374e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:40.115854 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17537713-092a-4e75-b0e3-227aa35374e8 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:40.116339 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17537713-092a-4e75-b0e3-227aa35374e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:40.116866 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17537713-092a-4e75-b0e3-227aa35374e8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:40.130118 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17537713-092a-4e75-b0e3-227aa35374e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:40.140700 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-17537713-092a-4e75-b0e3-227aa35374e8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 14:06:40.140700 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:40.140700 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:40.140700 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:40.140700 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:40.140700 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:40.140700 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:40.140700 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:40.140700 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:40.140700 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:40.140700 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:40.140700 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:40.140700 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:40.140700 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:40.140700 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:40.140700 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:40.141630 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:40.141630 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:40.141630 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:40.141630 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:40.141630 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:40.141630 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:40.141630 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:40.141630 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:40.141630 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:40.141630 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:40.141630 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:40.141630 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:40.141630 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:40.141630 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:40.141630 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:40.141630 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:40.142408 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:40.142408 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:40.142408 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:40.142408 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:40.142408 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:40.142408 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:40.142408 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:40.142408 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:40.142408 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:40.142408 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:40.142408 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 14:06:40.142408 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:06:40.145049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 118/227] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Aug 5 14:06:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:06:40.152021 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17537713-092a-4e75-b0e3-227aa35374e8 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:40.167580 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e2cbc-d0ae-4f13-be5c-57a4674996d3 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:40.214219 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:40.222980 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc59ac77-c8fc-4978-a216-9da8832bbbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:40.223629 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59ac77-c8fc-4978-a216-9da8832bbbf1 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:40.223904 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59ac77-c8fc-4978-a216-9da8832bbbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:40.224278 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59ac77-c8fc-4978-a216-9da8832bbbf1 None admin] PATH_INFO: `/v3/users/ffbe094b250840fa86b8b1ecf2ce214d` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:40.225919 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc59ac77-c8fc-4978-a216-9da8832bbbf1 None admin] RBAC: Authorizing `identity:get_user(user_id=ffbe094b250840fa86b8b1ecf2ce214d)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:40.229710 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc59ac77-c8fc-4978-a216-9da8832bbbf1 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:40.234672 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 110/228] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:06:39 2022] GET /identity/v3/users/ffbe094b250840fa86b8b1ecf2ce214d => generated 283 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:40.235869 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc59ac77-c8fc-4978-a216-9da8832bbbf1 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:40.304161 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17537713-092a-4e75-b0e3-227aa35374e8 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:40.364595 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da28c29-73f2-4213-9855-98a501afe467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:40.365760 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da28c29-73f2-4213-9855-98a501afe467 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:40.366456 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da28c29-73f2-4213-9855-98a501afe467 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:40.366939 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da28c29-73f2-4213-9855-98a501afe467 None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:40.367876 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da28c29-73f2-4213-9855-98a501afe467 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:40.370006 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da28c29-73f2-4213-9855-98a501afe467 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:40.371721 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da28c29-73f2-4213-9855-98a501afe467 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:40.381746 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:40.389915 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 119/229] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:06:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:40.398074 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da28c29-73f2-4213-9855-98a501afe467 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:40.432050 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc59ac77-c8fc-4978-a216-9da8832bbbf1 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:40.496302 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763983ce-ea3f-4284-8395-d975690d7238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:40.497351 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763983ce-ea3f-4284-8395-d975690d7238 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:40.497820 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763983ce-ea3f-4284-8395-d975690d7238 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:40.498615 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763983ce-ea3f-4284-8395-d975690d7238 None admin] PATH_INFO: `/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:40.500996 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763983ce-ea3f-4284-8395-d975690d7238 None admin] RBAC: Authorizing `identity:get_project(project_id=ecbbd8ae8eed4ce5b441221e8e17663b)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:40.502900 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763983ce-ea3f-4284-8395-d975690d7238 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:40.507587 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 111/230] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:06:40 2022] GET /identity/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b => generated 302 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:06:40.511764 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763983ce-ea3f-4284-8395-d975690d7238 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:40.590955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da28c29-73f2-4213-9855-98a501afe467 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:40.621970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:40.650024 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e80d01-5d1f-490e-8b26-93884b00818a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:40.650680 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e80d01-5d1f-490e-8b26-93884b00818a None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:40.650955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e80d01-5d1f-490e-8b26-93884b00818a None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:40.651225 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e80d01-5d1f-490e-8b26-93884b00818a None admin] PATH_INFO: `/v3/users/ffbe094b250840fa86b8b1ecf2ce214d` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:40.652489 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e80d01-5d1f-490e-8b26-93884b00818a None admin] RBAC: Authorizing `identity:get_user(user_id=ffbe094b250840fa86b8b1ecf2ce214d)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:40.658389 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e80d01-5d1f-490e-8b26-93884b00818a None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:40.660017 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 120/231] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:06:40 2022] GET /identity/v3/users/ffbe094b250840fa86b8b1ecf2ce214d => generated 283 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:40.667517 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e80d01-5d1f-490e-8b26-93884b00818a None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:40.689425 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763983ce-ea3f-4284-8395-d975690d7238 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:40.754634 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:40.777698 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb98a316-4ec7-4f8c-979d-638c1d572d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:40.778390 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98a316-4ec7-4f8c-979d-638c1d572d19 None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:40.778674 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98a316-4ec7-4f8c-979d-638c1d572d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:40.778948 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98a316-4ec7-4f8c-979d-638c1d572d19 None admin] PATH_INFO: `/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b/users/ffbe094b250840fa86b8b1ecf2ce214d/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:40.785707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb98a316-4ec7-4f8c-979d-638c1d572d19 None admin] RBAC: Authorizing `identity:create_grant(project_id=ecbbd8ae8eed4ce5b441221e8e17663b, user_id=ffbe094b250840fa86b8b1ecf2ce214d, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:40.787252 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb98a316-4ec7-4f8c-979d-638c1d572d19 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:40.793270 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e80d01-5d1f-490e-8b26-93884b00818a None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:40.830822 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24163308-ea25-40a8-aec0-a3bd0d94cb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:40.831520 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24163308-ea25-40a8-aec0-a3bd0d94cb12 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:40.831795 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24163308-ea25-40a8-aec0-a3bd0d94cb12 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:40.832114 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24163308-ea25-40a8-aec0-a3bd0d94cb12 None admin] PATH_INFO: `/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:40.833352 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24163308-ea25-40a8-aec0-a3bd0d94cb12 None admin] RBAC: Authorizing `identity:get_project(project_id=ecbbd8ae8eed4ce5b441221e8e17663b)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:40.835975 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24163308-ea25-40a8-aec0-a3bd0d94cb12 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:40.838004 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 121/232] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:06:40 2022] GET /identity/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:40.843316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24163308-ea25-40a8-aec0-a3bd0d94cb12 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:40.867325 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 112/233] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Aug 5 14:06:40 2022] PUT /identity/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b/users/ffbe094b250840fa86b8b1ecf2ce214d/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:06:40.981078 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24163308-ea25-40a8-aec0-a3bd0d94cb12 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:41.018437 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8fceb-5fd0-468e-8320-4652b0864e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:41.019091 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8fceb-5fd0-468e-8320-4652b0864e31 None admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:41.019429 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8fceb-5fd0-468e-8320-4652b0864e31 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:41.019707 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8fceb-5fd0-468e-8320-4652b0864e31 None admin] PATH_INFO: `/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b/users/ffbe094b250840fa86b8b1ecf2ce214d/roles/199fd3093a824ba585b1135c8bf363be` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:41.022610 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8fceb-5fd0-468e-8320-4652b0864e31 None admin] RBAC: Authorizing `identity:create_grant(project_id=ecbbd8ae8eed4ce5b441221e8e17663b, user_id=ffbe094b250840fa86b8b1ecf2ce214d, role_id=199fd3093a824ba585b1135c8bf363be)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:41.024066 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8fceb-5fd0-468e-8320-4652b0864e31 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:41.026704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:41.170639 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 122/234] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Aug 5 14:06:40 2022] PUT /identity/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b/users/ffbe094b250840fa86b8b1ecf2ce214d/roles/199fd3093a824ba585b1135c8bf363be => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:06:41.253214 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a998b-3bd7-4149-b677-6122bf3c54e2 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:41.253851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a998b-3bd7-4149-b677-6122bf3c54e2 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:41.253851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a998b-3bd7-4149-b677-6122bf3c54e2 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:41.255094 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 113/235] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:41.260968 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f65435-cd3e-4ae1-8826-bbb5e38bc298 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:41.261271 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f65435-cd3e-4ae1-8826-bbb5e38bc298 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:41.261506 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f65435-cd3e-4ae1-8826-bbb5e38bc298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:41.289787 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-64f65435-cd3e-4ae1-8826-bbb5e38bc298 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:41.291325 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-64f65435-cd3e-4ae1-8826-bbb5e38bc298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:41.389900 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 123/236] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 14:06:41.414092 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a998b-3bd7-4149-b677-6122bf3c54e2 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:41.463967 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f80011-a1f8-45b9-8b2a-a748b134b55c None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:41.464661 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f80011-a1f8-45b9-8b2a-a748b134b55c None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:41.465135 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f80011-a1f8-45b9-8b2a-a748b134b55c None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:41.466963 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 124/237] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:06:41.472234 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4d3c5-f5a0-4dfe-bbdf-f180219a02f7 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:41.472561 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4d3c5-f5a0-4dfe-bbdf-f180219a02f7 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:41.472845 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4d3c5-f5a0-4dfe-bbdf-f180219a02f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:41.505677 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-05c4d3c5-f5a0-4dfe-bbdf-f180219a02f7 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:41.507340 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-05c4d3c5-f5a0-4dfe-bbdf-f180219a02f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:41.541781 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a998b-3bd7-4149-b677-6122bf3c54e2 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:41.565737 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:41.597422 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3d49aa-be56-451d-99ea-1848760f5cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:41.598085 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d49aa-be56-451d-99ea-1848760f5cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:41.598365 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d49aa-be56-451d-99ea-1848760f5cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:41.598640 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d49aa-be56-451d-99ea-1848760f5cf6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:41.607364 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 125/238] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 14:06:41.621018 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3d49aa-be56-451d-99ea-1848760f5cf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:41.626489 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-fd3d49aa-be56-451d-99ea-1848760f5cf6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 14:06:41.626489 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:41.626489 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:41.626489 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:41.626489 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:41.626489 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:41.626489 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:41.626489 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:41.626489 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:41.626489 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:41.626489 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:41.626489 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:41.626489 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:41.626489 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:41.626489 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:41.626489 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:41.627374 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:41.627374 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:41.627374 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:41.627374 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:41.627374 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:41.627374 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:41.627374 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:41.627374 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:41.627374 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:41.627374 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:41.627374 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:41.627374 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:41.627374 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:41.627374 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:41.627374 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:41.627374 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:41.628158 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:41.628158 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:41.628158 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:41.628158 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:41.628158 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:41.628158 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:41.628158 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:41.628158 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:41.628158 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:41.628158 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:41.628158 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 14:06:41.628158 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:06:41.628158 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 114/239] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:06:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:06:41.634526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c4d3c5-f5a0-4dfe-bbdf-f180219a02f7 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:41.658483 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3d49aa-be56-451d-99ea-1848760f5cf6 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:41.722160 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:41.766625 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3d49aa-be56-451d-99ea-1848760f5cf6 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:41.794618 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c4d3c5-f5a0-4dfe-bbdf-f180219a02f7 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:41.803498 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb78edd6-7c05-44d1-8684-f8c4cc7b0463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:41.804136 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb78edd6-7c05-44d1-8684-f8c4cc7b0463 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:41.804422 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb78edd6-7c05-44d1-8684-f8c4cc7b0463 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:41.804696 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb78edd6-7c05-44d1-8684-f8c4cc7b0463 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:41.805934 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb78edd6-7c05-44d1-8684-f8c4cc7b0463 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:41.807436 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb78edd6-7c05-44d1-8684-f8c4cc7b0463 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:41.809327 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 115/240] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:06:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:41.814748 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb78edd6-7c05-44d1-8684-f8c4cc7b0463 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:41.853231 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe824a9-54cd-4304-bcc2-d4ce8b14fc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:41.853862 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe824a9-54cd-4304-bcc2-d4ce8b14fc77 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:41.854138 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe824a9-54cd-4304-bcc2-d4ce8b14fc77 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:41.854405 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe824a9-54cd-4304-bcc2-d4ce8b14fc77 None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:41.855125 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe824a9-54cd-4304-bcc2-d4ce8b14fc77 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:41.855425 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe824a9-54cd-4304-bcc2-d4ce8b14fc77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:41.856888 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe824a9-54cd-4304-bcc2-d4ce8b14fc77 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:41.865602 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 126/241] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Aug 5 14:06:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:41.872360 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe824a9-54cd-4304-bcc2-d4ce8b14fc77 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:41.876384 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:41.929371 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb78edd6-7c05-44d1-8684-f8c4cc7b0463 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:41.967411 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f637db2-37bb-4dbc-863e-79976be95f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:41.968031 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f637db2-37bb-4dbc-863e-79976be95f32 None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:41.968307 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f637db2-37bb-4dbc-863e-79976be95f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:41.968597 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f637db2-37bb-4dbc-863e-79976be95f32 None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:41.969293 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f637db2-37bb-4dbc-863e-79976be95f32 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:41.970727 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f637db2-37bb-4dbc-863e-79976be95f32 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:42.032667 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe824a9-54cd-4304-bcc2-d4ce8b14fc77 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:42.050082 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:42.058999 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 116/242] 10.222.0.26 () {60 vars in 1132 bytes} [Fri Aug 5 14:06:41 2022] POST /identity/v3/users => generated 293 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:06:42.087773 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83146736-1f14-4e75-ad2d-0fccc47e65dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:42.088625 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83146736-1f14-4e75-ad2d-0fccc47e65dd None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:42.088909 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83146736-1f14-4e75-ad2d-0fccc47e65dd None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:42.089186 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83146736-1f14-4e75-ad2d-0fccc47e65dd None admin] PATH_INFO: `/v3/users/441cf4b2b234489caab48aa696b8080a` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:42.105831 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83146736-1f14-4e75-ad2d-0fccc47e65dd None admin] RBAC: Authorizing `identity:get_user(user_id=441cf4b2b234489caab48aa696b8080a)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:42.107396 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83146736-1f14-4e75-ad2d-0fccc47e65dd None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:42.109504 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 127/243] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:06:41 2022] GET /identity/v3/users/441cf4b2b234489caab48aa696b8080a => generated 297 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:42.115011 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f637db2-37bb-4dbc-863e-79976be95f32 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:42.263791 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f637db2-37bb-4dbc-863e-79976be95f32 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:42.275211 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:42.306967 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc4439-1dcf-43ca-bd20-0129499e312e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:42.307591 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc4439-1dcf-43ca-bd20-0129499e312e None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:42.307933 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc4439-1dcf-43ca-bd20-0129499e312e None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:42.308203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc4439-1dcf-43ca-bd20-0129499e312e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:42.380500 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc4439-1dcf-43ca-bd20-0129499e312e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:42.382640 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc4439-1dcf-43ca-bd20-0129499e312e None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:42.493067 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 117/244] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:06:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 14:06:42.499266 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83146736-1f14-4e75-ad2d-0fccc47e65dd None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:42.616306 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83146736-1f14-4e75-ad2d-0fccc47e65dd None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:42.659207 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e9387-5f1b-4bf9-af52-9212affe4c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:42.659866 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e9387-5f1b-4bf9-af52-9212affe4c8d None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:42.660143 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e9387-5f1b-4bf9-af52-9212affe4c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:42.660441 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e9387-5f1b-4bf9-af52-9212affe4c8d None admin] PATH_INFO: `/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:42.661759 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e9387-5f1b-4bf9-af52-9212affe4c8d None admin] RBAC: Authorizing `identity:get_project(project_id=ecbbd8ae8eed4ce5b441221e8e17663b)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:42.663453 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e9387-5f1b-4bf9-af52-9212affe4c8d None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:42.665495 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 128/245] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:06:42 2022] GET /identity/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:06:42.670931 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc4439-1dcf-43ca-bd20-0129499e312e None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:42.679439 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:42.706845 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad170ff-00dd-4af6-b6e8-b0c5d30c5a95 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:42.707121 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad170ff-00dd-4af6-b6e8-b0c5d30c5a95 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:42.707394 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad170ff-00dd-4af6-b6e8-b0c5d30c5a95 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:42.708895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 129/246] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:06:42.714856 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431d66d-0398-47e4-b4cf-46c9822f3b22 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:42.715137 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431d66d-0398-47e4-b4cf-46c9822f3b22 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:42.715412 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431d66d-0398-47e4-b4cf-46c9822f3b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:42.735068 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-9431d66d-0398-47e4-b4cf-46c9822f3b22 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:42.736330 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-9431d66d-0398-47e4-b4cf-46c9822f3b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:42.774841 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:42.804755 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc4439-1dcf-43ca-bd20-0129499e312e None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:42.816783 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 130/247] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 14:06:42.841182 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9431d66d-0398-47e4-b4cf-46c9822f3b22 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:42.861699 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b2457b-45b3-4032-98b9-0ad993893b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:42.862328 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2457b-45b3-4032-98b9-0ad993893b90 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:42.862606 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2457b-45b3-4032-98b9-0ad993893b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:42.862933 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2457b-45b3-4032-98b9-0ad993893b90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:42.863724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b2457b-45b3-4032-98b9-0ad993893b90 None admin] RBAC: Adding query filter params (role.id=8e4d742669e54e288bac0e45718908b8, scope.project.id=ecbbd8ae8eed4ce5b441221e8e17663b, user.id=441cf4b2b234489caab48aa696b8080a) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:42.864074 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b2457b-45b3-4032-98b9-0ad993893b90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:42.866231 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b2457b-45b3-4032-98b9-0ad993893b90 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:42.876016 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 118/248] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:06:42 2022] GET /identity/v3/role_assignments?user.id=441cf4b2b234489caab48aa696b8080a&scope.project.id=ecbbd8ae8eed4ce5b441221e8e17663b&role.id=8e4d742669e54e288bac0e45718908b8 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:06:42.964350 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9431d66d-0398-47e4-b4cf-46c9822f3b22 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:43.007696 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0169392d-48d3-489c-82e4-76a3a7bd55ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:43.008636 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169392d-48d3-489c-82e4-76a3a7bd55ff None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:43.009087 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169392d-48d3-489c-82e4-76a3a7bd55ff None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:43.009501 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169392d-48d3-489c-82e4-76a3a7bd55ff None admin] PATH_INFO: `/v3/roles/member` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:43.021087 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0169392d-48d3-489c-82e4-76a3a7bd55ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:43.031303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-0169392d-48d3-489c-82e4-76a3a7bd55ff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 14:06:43.031303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:43.031303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:43.031303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:43.031303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:43.031303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:43.031303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:43.031303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:43.031303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:43.031303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:43.031303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:43.031303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:43.031303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:43.031303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:43.031303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:43.031303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:43.032461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:43.032461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:43.032461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:43.032461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:43.032461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:43.032461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:43.032461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:43.032461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:43.032461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:43.032461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:43.032461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:43.032461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:43.032461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:43.032461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:43.032461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:43.032461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:43.033907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:43.033907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:43.033907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:43.033907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:43.033907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:43.033907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:43.033907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:43.033907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:43.033907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:43.033907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:43.033907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 14:06:43.033907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:06:43.035992 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 131/249] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:06:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:06:43.042564 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b2457b-45b3-4032-98b9-0ad993893b90 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:43.128633 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:43.169906 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b2457b-45b3-4032-98b9-0ad993893b90 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:43.218049 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49244dc2-da7d-4ab1-8de3-94c7e1c8dfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:43.218680 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49244dc2-da7d-4ab1-8de3-94c7e1c8dfcc None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:43.218962 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49244dc2-da7d-4ab1-8de3-94c7e1c8dfcc None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:43.219236 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49244dc2-da7d-4ab1-8de3-94c7e1c8dfcc None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:43.219958 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49244dc2-da7d-4ab1-8de3-94c7e1c8dfcc None admin] RBAC: Adding query filter params (name=member) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:43.220263 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49244dc2-da7d-4ab1-8de3-94c7e1c8dfcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:43.221678 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49244dc2-da7d-4ab1-8de3-94c7e1c8dfcc None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:43.236090 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 119/250] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Aug 5 14:06:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:43.242299 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0169392d-48d3-489c-82e4-76a3a7bd55ff None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:43.289920 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:43.376845 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0169392d-48d3-489c-82e4-76a3a7bd55ff None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:43.415777 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbfe49d-afae-4603-97d9-65dfa18f4767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:43.416334 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfe49d-afae-4603-97d9-65dfa18f4767 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:43.416677 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfe49d-afae-4603-97d9-65dfa18f4767 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:43.416946 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfe49d-afae-4603-97d9-65dfa18f4767 None admin] PATH_INFO: `/v3/users/182e7de0d85f40f0941829018ce80365` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:43.418121 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbfe49d-afae-4603-97d9-65dfa18f4767 None admin] RBAC: Authorizing `identity:get_user(user_id=182e7de0d85f40f0941829018ce80365)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:43.419546 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbfe49d-afae-4603-97d9-65dfa18f4767 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:43.421460 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 132/251] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:06:43 2022] GET /identity/v3/users/182e7de0d85f40f0941829018ce80365 => generated 297 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:43.429802 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49244dc2-da7d-4ab1-8de3-94c7e1c8dfcc None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:43.469397 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13b379-345e-4089-bf1d-08352c096a9c None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:43.469397 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13b379-345e-4089-bf1d-08352c096a9c None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:43.469397 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13b379-345e-4089-bf1d-08352c096a9c None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:43.470515 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 133/252] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:06:43.493610 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2117155-352a-46f7-b869-c05813af51bd None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:43.494027 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2117155-352a-46f7-b869-c05813af51bd None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:43.500437 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2117155-352a-46f7-b869-c05813af51bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:43.553629 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-f2117155-352a-46f7-b869-c05813af51bd None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:43.554858 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-f2117155-352a-46f7-b869-c05813af51bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:43.620700 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49244dc2-da7d-4ab1-8de3-94c7e1c8dfcc None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:43.676853 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:43.685595 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e915969c-242a-4c5d-bda4-69e136f70e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:43.694967 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915969c-242a-4c5d-bda4-69e136f70e96 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:43.694967 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915969c-242a-4c5d-bda4-69e136f70e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:43.694967 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915969c-242a-4c5d-bda4-69e136f70e96 None admin] PATH_INFO: `/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:43.694967 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e915969c-242a-4c5d-bda4-69e136f70e96 None admin] RBAC: Authorizing `identity:get_project(project_id=ecbbd8ae8eed4ce5b441221e8e17663b)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:43.694967 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e915969c-242a-4c5d-bda4-69e136f70e96 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:43.694967 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 120/253] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:06:43 2022] GET /identity/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b => generated 302 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:43.696096 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cebd54-1273-4296-a7bb-59d4d38e2c9f None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:43.696386 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cebd54-1273-4296-a7bb-59d4d38e2c9f None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:43.703497 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cebd54-1273-4296-a7bb-59d4d38e2c9f None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:43.705262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 121/254] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:43 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:43.705992 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cebd54-1273-4296-a7bb-59d4d38e2c9f None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:43.726982 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 134/255] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 05 14:06:43.729249 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4841e66-2f42-4a89-aa62-a5c1787a422a None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:43.729951 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4841e66-2f42-4a89-aa62-a5c1787a422a None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:43.731572 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4841e66-2f42-4a89-aa62-a5c1787a422a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:43.763931 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-b4841e66-2f42-4a89-aa62-a5c1787a422a None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:43.765468 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-b4841e66-2f42-4a89-aa62-a5c1787a422a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:43.814862 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:43.890969 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 135/256] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:43.892889 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4841e66-2f42-4a89-aa62-a5c1787a422a None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:43.911425 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cebd54-1273-4296-a7bb-59d4d38e2c9f None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:43.979224 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8bc77-d65e-4897-8f24-781f325f6009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:43.980256 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8bc77-d65e-4897-8f24-781f325f6009 None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:43.980772 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8bc77-d65e-4897-8f24-781f325f6009 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:43.982057 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8bc77-d65e-4897-8f24-781f325f6009 None admin] PATH_INFO: `/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b/users/182e7de0d85f40f0941829018ce80365/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:43.986894 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b8bc77-d65e-4897-8f24-781f325f6009 None admin] RBAC: Authorizing `identity:create_grant(project_id=ecbbd8ae8eed4ce5b441221e8e17663b, user_id=182e7de0d85f40f0941829018ce80365, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:43.988627 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b8bc77-d65e-4897-8f24-781f325f6009 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:44.039718 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4841e66-2f42-4a89-aa62-a5c1787a422a None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:44.060862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 122/257] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Aug 5 14:06:43 2022] PUT /identity/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b/users/182e7de0d85f40f0941829018ce80365/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:06:44.062037 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8bc77-d65e-4897-8f24-781f325f6009 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:44.114503 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427b8bd-215c-4146-b650-190227ad5cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:44.116809 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427b8bd-215c-4146-b650-190227ad5cbf None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:44.118277 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427b8bd-215c-4146-b650-190227ad5cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:44.119881 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427b8bd-215c-4146-b650-190227ad5cbf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:44.144248 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b427b8bd-215c-4146-b650-190227ad5cbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:44.150702 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-b427b8bd-215c-4146-b650-190227ad5cbf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:06:44.150702 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:44.150702 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:44.150702 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:44.150702 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:44.150702 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:44.150702 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:44.150702 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:44.150702 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:44.150702 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:44.150702 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:44.150702 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:44.150702 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:44.150702 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:44.150702 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:44.150702 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:44.151679 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:44.151679 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:44.151679 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:44.151679 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:44.151679 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:44.151679 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:44.151679 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:44.151679 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:44.151679 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:44.151679 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:44.151679 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:44.151679 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:44.151679 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:44.151679 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:44.151679 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:44.151679 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:44.152514 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:44.152514 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:44.152514 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:44.152514 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:44.152514 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:44.152514 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:44.152514 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:44.152514 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:44.152514 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:44.152514 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:44.152514 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:06:44.152514 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:06:44.152514 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 136/258] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Aug 5 14:06:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:06:44.161722 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427b8bd-215c-4146-b650-190227ad5cbf None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:44.187207 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:44.200844 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:44.271043 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8bc77-d65e-4897-8f24-781f325f6009 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:44.312255 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df006bb-1a48-44da-8299-289e9d9669ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:44.312999 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df006bb-1a48-44da-8299-289e9d9669ae None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:44.313279 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df006bb-1a48-44da-8299-289e9d9669ae None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:44.313600 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df006bb-1a48-44da-8299-289e9d9669ae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:44.325383 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df006bb-1a48-44da-8299-289e9d9669ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:44.339045 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-7df006bb-1a48-44da-8299-289e9d9669ae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 14:06:44.339045 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:44.339045 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:44.339045 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:44.339045 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:44.339045 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:44.339045 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:44.339045 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:44.339045 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:44.339045 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:44.339045 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:44.339045 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:44.339045 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:44.339045 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:44.339045 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:44.339045 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:44.339984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:44.339984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:44.339984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:44.339984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:44.339984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:44.339984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:44.339984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:44.339984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:44.339984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:44.339984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:44.339984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:44.339984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:44.339984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:44.339984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:44.339984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:44.339984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:44.340832 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:44.340832 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:44.340832 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:44.340832 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:44.340832 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:44.340832 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:44.340832 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:44.340832 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:44.340832 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:44.340832 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:44.340832 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 14:06:44.340832 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:06:44.340832 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 123/259] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Aug 5 14:06:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:06:44.358580 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df006bb-1a48-44da-8299-289e9d9669ae None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:44.379648 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427b8bd-215c-4146-b650-190227ad5cbf None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:44.480008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e175e09-8e95-4b58-9343-ef2af6839a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:44.481320 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e175e09-8e95-4b58-9343-ef2af6839a19 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:44.481320 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e175e09-8e95-4b58-9343-ef2af6839a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:44.481320 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e175e09-8e95-4b58-9343-ef2af6839a19 None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:44.482031 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e175e09-8e95-4b58-9343-ef2af6839a19 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:44.482376 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e175e09-8e95-4b58-9343-ef2af6839a19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:44.485991 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e175e09-8e95-4b58-9343-ef2af6839a19 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:44.502624 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df006bb-1a48-44da-8299-289e9d9669ae None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:44.523201 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 137/260] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:06:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:06:44.538201 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e175e09-8e95-4b58-9343-ef2af6839a19 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:44.544200 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:44.558454 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5227ef63-3255-44dd-a933-9d09cb7d7bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:44.559148 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227ef63-3255-44dd-a933-9d09cb7d7bde None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:44.559430 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227ef63-3255-44dd-a933-9d09cb7d7bde None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:44.559704 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227ef63-3255-44dd-a933-9d09cb7d7bde None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:44.560495 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5227ef63-3255-44dd-a933-9d09cb7d7bde None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:44.560858 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5227ef63-3255-44dd-a933-9d09cb7d7bde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:44.562241 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5227ef63-3255-44dd-a933-9d09cb7d7bde None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:44.572895 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 124/261] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:06:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:44.579248 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5227ef63-3255-44dd-a933-9d09cb7d7bde None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:44.689859 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5227ef63-3255-44dd-a933-9d09cb7d7bde None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:44.745861 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada2165-b8f9-45a7-9f1a-2a8d40475a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:44.746524 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada2165-b8f9-45a7-9f1a-2a8d40475a79 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:44.746802 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada2165-b8f9-45a7-9f1a-2a8d40475a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:44.747070 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada2165-b8f9-45a7-9f1a-2a8d40475a79 None admin] PATH_INFO: `/v3/users/ffbe094b250840fa86b8b1ecf2ce214d` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:44.748549 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bada2165-b8f9-45a7-9f1a-2a8d40475a79 None admin] RBAC: Authorizing `identity:get_user(user_id=ffbe094b250840fa86b8b1ecf2ce214d)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:44.750124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bada2165-b8f9-45a7-9f1a-2a8d40475a79 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:44.760942 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 125/262] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:06:44 2022] GET /identity/v3/users/ffbe094b250840fa86b8b1ecf2ce214d => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:44.766573 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada2165-b8f9-45a7-9f1a-2a8d40475a79 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:44.769459 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:44.803576 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e175e09-8e95-4b58-9343-ef2af6839a19 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:44.887540 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:44.941237 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a10064-2b81-4138-a787-9107ec69c3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:44.941237 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a10064-2b81-4138-a787-9107ec69c3fb None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:44.941237 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a10064-2b81-4138-a787-9107ec69c3fb None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:44.941237 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a10064-2b81-4138-a787-9107ec69c3fb None admin] PATH_INFO: `/v3/users/ffbe094b250840fa86b8b1ecf2ce214d` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:44.941653 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a10064-2b81-4138-a787-9107ec69c3fb None admin] RBAC: Authorizing `identity:get_user(user_id=ffbe094b250840fa86b8b1ecf2ce214d)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:44.941653 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a10064-2b81-4138-a787-9107ec69c3fb None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:44.952625 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 138/263] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:06:44 2022] GET /identity/v3/users/ffbe094b250840fa86b8b1ecf2ce214d => generated 283 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:44.952625 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a10064-2b81-4138-a787-9107ec69c3fb None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:44.999298 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada2165-b8f9-45a7-9f1a-2a8d40475a79 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:45.085967 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8394617-b5af-422d-b5e0-c68eb30b0205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:45.086519 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8394617-b5af-422d-b5e0-c68eb30b0205 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:45.086793 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8394617-b5af-422d-b5e0-c68eb30b0205 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:45.087057 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8394617-b5af-422d-b5e0-c68eb30b0205 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:45.197852 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a10064-2b81-4138-a787-9107ec69c3fb None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:45.241526 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8394617-b5af-422d-b5e0-c68eb30b0205 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:45.242998 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8394617-b5af-422d-b5e0-c68eb30b0205 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:45.258540 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb80a41-44c4-48eb-b46f-760c51f99967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:45.259177 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb80a41-44c4-48eb-b46f-760c51f99967 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:45.259456 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb80a41-44c4-48eb-b46f-760c51f99967 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:45.259725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb80a41-44c4-48eb-b46f-760c51f99967 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:45.304088 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:45.340852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:45.368102 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb80a41-44c4-48eb-b46f-760c51f99967 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:45.369614 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb80a41-44c4-48eb-b46f-760c51f99967 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:45.419806 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 126/264] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:06:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 654 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 14:06:45.429491 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8394617-b5af-422d-b5e0-c68eb30b0205 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:45.508736 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 139/265] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:06:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 14:06:45.514289 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb80a41-44c4-48eb-b46f-760c51f99967 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:45.574127 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8394617-b5af-422d-b5e0-c68eb30b0205 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:45.608858 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:45.661237 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd935dac-2f95-4ab1-8173-6306d3e592f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:45.661896 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd935dac-2f95-4ab1-8173-6306d3e592f0 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:45.662227 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd935dac-2f95-4ab1-8173-6306d3e592f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:45.662496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd935dac-2f95-4ab1-8173-6306d3e592f0 None admin] PATH_INFO: `/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:45.663681 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd935dac-2f95-4ab1-8173-6306d3e592f0 None admin] RBAC: Authorizing `identity:get_project(project_id=ecbbd8ae8eed4ce5b441221e8e17663b)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:45.665162 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd935dac-2f95-4ab1-8173-6306d3e592f0 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:45.672854 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 127/266] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:06:45 2022] GET /identity/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:45.678810 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd935dac-2f95-4ab1-8173-6306d3e592f0 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:45.706524 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb80a41-44c4-48eb-b46f-760c51f99967 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:45.714921 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:45.777804 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3bfffd-fcb3-4cf4-8f2e-42c7e7f1d6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:45.778839 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3bfffd-fcb3-4cf4-8f2e-42c7e7f1d6cd None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:45.779426 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3bfffd-fcb3-4cf4-8f2e-42c7e7f1d6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:45.779951 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3bfffd-fcb3-4cf4-8f2e-42c7e7f1d6cd None admin] PATH_INFO: `/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:45.783298 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3bfffd-fcb3-4cf4-8f2e-42c7e7f1d6cd None admin] RBAC: Authorizing `identity:get_project(project_id=ecbbd8ae8eed4ce5b441221e8e17663b)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:45.785010 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3bfffd-fcb3-4cf4-8f2e-42c7e7f1d6cd None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:45.787186 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 140/267] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:06:45 2022] GET /identity/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b => generated 302 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:45.788622 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe7742-46f1-460f-a26c-ac6d225ce95e None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:45.789118 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe7742-46f1-460f-a26c-ac6d225ce95e None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:45.789608 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe7742-46f1-460f-a26c-ac6d225ce95e None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:45.791040 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 141/268] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:45.797433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe7742-46f1-460f-a26c-ac6d225ce95e None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:45.862364 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd935dac-2f95-4ab1-8173-6306d3e592f0 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:45.924214 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc65b1-735a-4f5a-9967-21445748e7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:45.924865 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc65b1-735a-4f5a-9967-21445748e7ed None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:45.925142 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc65b1-735a-4f5a-9967-21445748e7ed None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:45.925410 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc65b1-735a-4f5a-9967-21445748e7ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:45.926195 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc65b1-735a-4f5a-9967-21445748e7ed None admin] RBAC: Adding query filter params (role.id=199fd3093a824ba585b1135c8bf363be, scope.project.id=ecbbd8ae8eed4ce5b441221e8e17663b, user.id=ffbe094b250840fa86b8b1ecf2ce214d) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:45.926499 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc65b1-735a-4f5a-9967-21445748e7ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:45.930604 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc65b1-735a-4f5a-9967-21445748e7ed None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:45.946695 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 128/269] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:06:45 2022] GET /identity/v3/role_assignments?user.id=ffbe094b250840fa86b8b1ecf2ce214d&scope.project.id=ecbbd8ae8eed4ce5b441221e8e17663b&role.id=199fd3093a824ba585b1135c8bf363be => generated 608 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:45.947942 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7431d-da74-4d8d-8d5b-a9a4ea15f8d0 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:45.948229 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7431d-da74-4d8d-8d5b-a9a4ea15f8d0 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:45.964656 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7431d-da74-4d8d-8d5b-a9a4ea15f8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:45.996461 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-5cf7431d-da74-4d8d-8d5b-a9a4ea15f8d0 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:45.997972 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5cf7431d-da74-4d8d-8d5b-a9a4ea15f8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:46.001904 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe7742-46f1-460f-a26c-ac6d225ce95e None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:46.023463 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:46.050481 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831b12e6-f485-4e47-8db0-b858094563e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:46.051118 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b12e6-f485-4e47-8db0-b858094563e4 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:46.051392 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b12e6-f485-4e47-8db0-b858094563e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:46.051657 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b12e6-f485-4e47-8db0-b858094563e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:46.052502 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831b12e6-f485-4e47-8db0-b858094563e4 None admin] RBAC: Adding query filter params (role.id=299c794c7acb4a5ca36288c3191aad10, scope.project.id=ecbbd8ae8eed4ce5b441221e8e17663b, user.id=ffbe094b250840fa86b8b1ecf2ce214d) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:46.052842 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831b12e6-f485-4e47-8db0-b858094563e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:46.054385 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831b12e6-f485-4e47-8db0-b858094563e4 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:46.061454 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 142/270] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:06:45 2022] GET /identity/v3/role_assignments?user.id=ffbe094b250840fa86b8b1ecf2ce214d&scope.project.id=ecbbd8ae8eed4ce5b441221e8e17663b&role.id=299c794c7acb4a5ca36288c3191aad10 => generated 608 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:46.062950 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d37af7-48f4-4653-ae90-eb671c5a342d None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:46.063229 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d37af7-48f4-4653-ae90-eb671c5a342d None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:46.063506 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d37af7-48f4-4653-ae90-eb671c5a342d None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:46.065206 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 143/271] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:46.066439 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d6ae7-2a3f-4eea-be12-0efa0b16bbcd None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:46.066720 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d6ae7-2a3f-4eea-be12-0efa0b16bbcd None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:46.067033 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d6ae7-2a3f-4eea-be12-0efa0b16bbcd None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:46.068706 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 144/272] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:46.074339 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acf9c50-3e7e-4d8c-953d-e722400b191c None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:46.074710 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acf9c50-3e7e-4d8c-953d-e722400b191c None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:46.074935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acf9c50-3e7e-4d8c-953d-e722400b191c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:46.112846 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-8acf9c50-3e7e-4d8c-953d-e722400b191c None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:46.115545 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-8acf9c50-3e7e-4d8c-953d-e722400b191c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:46.141578 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 129/273] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:46.142886 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46fc80-d7cc-43db-86e1-e36dcb75c639 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:46.143173 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46fc80-d7cc-43db-86e1-e36dcb75c639 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:46.143437 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46fc80-d7cc-43db-86e1-e36dcb75c639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:46.173360 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-2a46fc80-d7cc-43db-86e1-e36dcb75c639 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:46.174604 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-2a46fc80-d7cc-43db-86e1-e36dcb75c639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:46.193651 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:46.240780 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 145/274] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:46.241735 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acf9c50-3e7e-4d8c-953d-e722400b191c None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:46.281268 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 130/275] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:46.285456 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a46fc80-d7cc-43db-86e1-e36dcb75c639 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:46.367616 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acf9c50-3e7e-4d8c-953d-e722400b191c None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:46.387677 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:46.412837 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812c928-ff9d-4b11-b154-d2f11011b9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:46.413479 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812c928-ff9d-4b11-b154-d2f11011b9e7 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:46.413757 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812c928-ff9d-4b11-b154-d2f11011b9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:46.414026 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812c928-ff9d-4b11-b154-d2f11011b9e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:46.415191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812c928-ff9d-4b11-b154-d2f11011b9e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:46.416654 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812c928-ff9d-4b11-b154-d2f11011b9e7 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:46.418465 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 146/276] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:06:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:46.419073 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812c928-ff9d-4b11-b154-d2f11011b9e7 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:46.430326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a46fc80-d7cc-43db-86e1-e36dcb75c639 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:46.464985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb4b48-04d8-428c-bfa4-7eb006afcd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:46.465627 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb4b48-04d8-428c-bfa4-7eb006afcd5c None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:46.465907 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb4b48-04d8-428c-bfa4-7eb006afcd5c None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:46.466182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb4b48-04d8-428c-bfa4-7eb006afcd5c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:46.476246 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb4b48-04d8-428c-bfa4-7eb006afcd5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:46.481262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-6deb4b48-04d8-428c-bfa4-7eb006afcd5c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 14:06:46.481262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:46.481262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:46.481262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:46.481262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:46.481262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:46.481262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:46.481262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:46.481262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:46.481262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:46.481262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:46.481262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:46.481262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:46.481262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:46.481262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:46.481262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:46.482149 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:46.482149 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:46.482149 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:46.482149 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:46.482149 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:46.482149 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:46.482149 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:46.482149 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:46.482149 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:46.482149 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:46.482149 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:46.482149 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:46.482149 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:46.482149 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:46.482149 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:46.482149 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:46.483205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:46.483205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:46.483205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:46.483205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:46.483205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:46.483205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:46.483205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:46.483205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:46.483205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:46.483205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:46.483205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 14:06:46.483205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:06:46.483205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 131/277] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:06:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:06:46.485294 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb4b48-04d8-428c-bfa4-7eb006afcd5c None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:46.488734 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:46.536270 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812c928-ff9d-4b11-b154-d2f11011b9e7 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:46.573241 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16df8756-22f5-4fd6-af14-953bd398764a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:46.574117 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df8756-22f5-4fd6-af14-953bd398764a None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:46.574986 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df8756-22f5-4fd6-af14-953bd398764a None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:46.575459 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df8756-22f5-4fd6-af14-953bd398764a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:46.591093 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16df8756-22f5-4fd6-af14-953bd398764a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:46.598187 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb4b48-04d8-428c-bfa4-7eb006afcd5c None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:46.598709 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-16df8756-22f5-4fd6-af14-953bd398764a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 14:06:46.598709 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:46.598709 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:46.598709 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:46.598709 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:46.598709 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:46.598709 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:46.598709 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:46.598709 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:46.598709 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:46.598709 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:46.598709 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:46.598709 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:46.598709 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:46.598709 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:46.598709 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:46.607665 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:46.607665 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:46.607665 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:46.607665 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:46.607665 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:46.607665 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:46.607665 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:46.607665 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:46.607665 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:46.607665 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:46.607665 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:46.607665 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:46.607665 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:46.607665 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:46.607665 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:46.607665 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:46.608469 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:46.608469 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:46.608469 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:46.608469 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:46.608469 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:46.608469 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:46.608469 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:46.608469 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:46.608469 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:46.608469 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:46.608469 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 14:06:46.608469 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:06:46.608469 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 147/278] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:06:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:06:46.608469 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16df8756-22f5-4fd6-af14-953bd398764a None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:46.637037 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9a58f-92d8-41ac-aa11-b338e1a40768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:46.637632 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9a58f-92d8-41ac-aa11-b338e1a40768 None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:46.637954 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9a58f-92d8-41ac-aa11-b338e1a40768 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:46.638220 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9a58f-92d8-41ac-aa11-b338e1a40768 None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:46.638914 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f9a58f-92d8-41ac-aa11-b338e1a40768 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:46.640342 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f9a58f-92d8-41ac-aa11-b338e1a40768 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:46.651403 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:46.707785 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16df8756-22f5-4fd6-af14-953bd398764a None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:46.743026 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aee2c1-310d-446b-a162-7fa28747e2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:46.743667 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aee2c1-310d-446b-a162-7fa28747e2ce None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:46.743947 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aee2c1-310d-446b-a162-7fa28747e2ce None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:46.744219 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aee2c1-310d-446b-a162-7fa28747e2ce None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:46.744973 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aee2c1-310d-446b-a162-7fa28747e2ce None admin] RBAC: Adding query filter params (name=member) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:46.745281 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aee2c1-310d-446b-a162-7fa28747e2ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:46.746650 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aee2c1-310d-446b-a162-7fa28747e2ce None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:46.754896 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 132/279] 10.222.0.26 () {60 vars in 1132 bytes} [Fri Aug 5 14:06:46 2022] POST /identity/v3/users => generated 293 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:06:46.754896 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9a58f-92d8-41ac-aa11-b338e1a40768 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:46.760993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 148/280] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Aug 5 14:06:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:46.766978 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aee2c1-310d-446b-a162-7fa28747e2ce None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:46.839664 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:46.877287 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aee2c1-310d-446b-a162-7fa28747e2ce None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:46.879258 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:46.896001 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9a58f-92d8-41ac-aa11-b338e1a40768 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:46.918311 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd8d2fe-4d7e-4ba2-933e-add0b87621df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:46.918967 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd8d2fe-4d7e-4ba2-933e-add0b87621df None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:46.919243 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd8d2fe-4d7e-4ba2-933e-add0b87621df None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:46.919516 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd8d2fe-4d7e-4ba2-933e-add0b87621df None admin] PATH_INFO: `/v3/users/1863a3bc116f4814a3e10925dbfba498` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:46.936583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd8d2fe-4d7e-4ba2-933e-add0b87621df None admin] RBAC: Authorizing `identity:get_user(user_id=1863a3bc116f4814a3e10925dbfba498)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:46.938057 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd8d2fe-4d7e-4ba2-933e-add0b87621df None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:46.939764 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567fc4ad-a99f-45bf-9c93-cd86b28e3b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:46.940057 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567fc4ad-a99f-45bf-9c93-cd86b28e3b84 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:46.940154 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 149/281] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:06:46 2022] GET /identity/v3/users/1863a3bc116f4814a3e10925dbfba498 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:46.941939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567fc4ad-a99f-45bf-9c93-cd86b28e3b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:46.942045 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c404533-2aa8-4c79-926f-bce49d9bce08 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:46.942131 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567fc4ad-a99f-45bf-9c93-cd86b28e3b84 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:46.942208 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c404533-2aa8-4c79-926f-bce49d9bce08 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:46.942208 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c404533-2aa8-4c79-926f-bce49d9bce08 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:46.942327 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567fc4ad-a99f-45bf-9c93-cd86b28e3b84 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:46.942516 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567fc4ad-a99f-45bf-9c93-cd86b28e3b84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:46.945954 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 150/282] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:46.946086 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567fc4ad-a99f-45bf-9c93-cd86b28e3b84 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:46.949219 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6509ef71-e5d1-4963-9c4b-ab2a6af64cda None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:46.949505 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6509ef71-e5d1-4963-9c4b-ab2a6af64cda None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:46.949780 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6509ef71-e5d1-4963-9c4b-ab2a6af64cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:46.954154 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 133/283] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Aug 5 14:06:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:46.954906 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567fc4ad-a99f-45bf-9c93-cd86b28e3b84 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:46.976643 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-6509ef71-e5d1-4963-9c4b-ab2a6af64cda None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:46.977865 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-6509ef71-e5d1-4963-9c4b-ab2a6af64cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:47.068779 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 151/284] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:47.068779 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6509ef71-e5d1-4963-9c4b-ab2a6af64cda None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:47.087798 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567fc4ad-a99f-45bf-9c93-cd86b28e3b84 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:47.116528 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:47.134691 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc92245-1b0f-4e43-b143-d5d9aa0530fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:47.135543 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc92245-1b0f-4e43-b143-d5d9aa0530fa None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:47.135966 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc92245-1b0f-4e43-b143-d5d9aa0530fa None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:47.136404 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc92245-1b0f-4e43-b143-d5d9aa0530fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:47.137360 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc92245-1b0f-4e43-b143-d5d9aa0530fa None admin] RBAC: Adding query filter params (role.id=29967df108484a30a4c006cde2ae2940, user.id=1863a3bc116f4814a3e10925dbfba498, scope.system=all) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:47.137814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc92245-1b0f-4e43-b143-d5d9aa0530fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:47.139471 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc92245-1b0f-4e43-b143-d5d9aa0530fa None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:47.145762 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 134/285] 10.222.0.26 () {56 vars in 1308 bytes} [Fri Aug 5 14:06:46 2022] GET /identity/v3/role_assignments?user.id=1863a3bc116f4814a3e10925dbfba498&scope.system=all&role.id=29967df108484a30a4c006cde2ae2940 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:47.147558 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc92245-1b0f-4e43-b143-d5d9aa0530fa None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:47.234741 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:47.284244 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6509ef71-e5d1-4963-9c4b-ab2a6af64cda None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:47.292849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc92245-1b0f-4e43-b143-d5d9aa0530fa None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:47.332810 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e19d3ce-03a9-42b0-9087-93413c4a00be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:47.333478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19d3ce-03a9-42b0-9087-93413c4a00be None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:47.334378 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19d3ce-03a9-42b0-9087-93413c4a00be None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:47.335436 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19d3ce-03a9-42b0-9087-93413c4a00be None admin] PATH_INFO: `/v3/roles/member` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:47.347417 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e19d3ce-03a9-42b0-9087-93413c4a00be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:47.354629 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-1e19d3ce-03a9-42b0-9087-93413c4a00be None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 14:06:47.354629 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:47.354629 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:47.354629 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:47.354629 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:47.354629 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:47.354629 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:47.354629 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:47.354629 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:47.354629 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:47.354629 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:47.354629 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:47.354629 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:47.354629 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:47.354629 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:47.354629 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:47.355777 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:47.355777 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:47.355777 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:47.355777 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:47.355777 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:47.355777 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:47.355777 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:47.355777 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:47.355777 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:47.355777 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:47.355777 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:47.355777 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:47.355777 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:47.355777 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:47.355777 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:47.355777 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:47.356823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:47.356823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:47.356823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:47.356823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:47.356823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:47.356823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:47.356823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:47.356823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:47.356823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:47.356823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:47.356823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 14:06:47.356823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:06:47.356823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 135/286] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:06:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:06:47.362878 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e19d3ce-03a9-42b0-9087-93413c4a00be None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:47.387291 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5850780f-2671-4a9d-b5f7-08679458adc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:47.388185 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850780f-2671-4a9d-b5f7-08679458adc7 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:47.388761 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850780f-2671-4a9d-b5f7-08679458adc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:47.389174 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850780f-2671-4a9d-b5f7-08679458adc7 None admin] PATH_INFO: `/v3/users/441cf4b2b234489caab48aa696b8080a` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:47.390571 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5850780f-2671-4a9d-b5f7-08679458adc7 None admin] RBAC: Authorizing `identity:get_user(user_id=441cf4b2b234489caab48aa696b8080a)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:47.392139 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5850780f-2671-4a9d-b5f7-08679458adc7 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:47.395869 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 152/287] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:06:47 2022] GET /identity/v3/users/441cf4b2b234489caab48aa696b8080a => generated 297 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:47.402298 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5850780f-2671-4a9d-b5f7-08679458adc7 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:47.422006 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:47.539812 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e19d3ce-03a9-42b0-9087-93413c4a00be None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:47.593434 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5850780f-2671-4a9d-b5f7-08679458adc7 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:47.593627 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec4bc1-823c-4446-883d-56fd97c680a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:47.594254 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec4bc1-823c-4446-883d-56fd97c680a3 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:47.594538 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec4bc1-823c-4446-883d-56fd97c680a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:47.594809 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec4bc1-823c-4446-883d-56fd97c680a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:47.595632 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ec4bc1-823c-4446-883d-56fd97c680a3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:47.595977 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ec4bc1-823c-4446-883d-56fd97c680a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:47.597461 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ec4bc1-823c-4446-883d-56fd97c680a3 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:47.612358 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 136/288] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Aug 5 14:06:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:06:47.621690 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec4bc1-823c-4446-883d-56fd97c680a3 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:47.636983 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1baaec1-dabb-4a80-a092-b8bac076785b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:47.637695 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1baaec1-dabb-4a80-a092-b8bac076785b None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:47.637975 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1baaec1-dabb-4a80-a092-b8bac076785b None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:47.638249 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1baaec1-dabb-4a80-a092-b8bac076785b None admin] PATH_INFO: `/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:47.639534 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1baaec1-dabb-4a80-a092-b8bac076785b None admin] RBAC: Authorizing `identity:get_project(project_id=ecbbd8ae8eed4ce5b441221e8e17663b)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:47.642940 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1baaec1-dabb-4a80-a092-b8bac076785b None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:47.649597 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 153/289] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:06:47 2022] GET /identity/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:06:47.656833 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1baaec1-dabb-4a80-a092-b8bac076785b None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:47.719583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:47.830421 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec4bc1-823c-4446-883d-56fd97c680a3 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:47.839966 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1baaec1-dabb-4a80-a092-b8bac076785b None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:47.847839 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:47.884062 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c763332-eb50-4bb5-9dc8-05767848a2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:47.884766 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c763332-eb50-4bb5-9dc8-05767848a2ad None admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:47.885139 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c763332-eb50-4bb5-9dc8-05767848a2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:47.885418 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c763332-eb50-4bb5-9dc8-05767848a2ad None admin] PATH_INFO: `/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b/users/441cf4b2b234489caab48aa696b8080a/roles/8e4d742669e54e288bac0e45718908b8` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:47.890002 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c763332-eb50-4bb5-9dc8-05767848a2ad None admin] RBAC: Authorizing `identity:create_grant(project_id=ecbbd8ae8eed4ce5b441221e8e17663b, user_id=441cf4b2b234489caab48aa696b8080a, role_id=8e4d742669e54e288bac0e45718908b8)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:47.891455 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c763332-eb50-4bb5-9dc8-05767848a2ad None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:47.905794 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3366c37f-c416-481a-9d8a-4d9d7bcfb2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:47.906667 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366c37f-c416-481a-9d8a-4d9d7bcfb2b6 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:47.907005 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366c37f-c416-481a-9d8a-4d9d7bcfb2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:47.907280 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366c37f-c416-481a-9d8a-4d9d7bcfb2b6 None admin] PATH_INFO: `/v3/users/182e7de0d85f40f0941829018ce80365` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:47.908583 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3366c37f-c416-481a-9d8a-4d9d7bcfb2b6 None admin] RBAC: Authorizing `identity:get_user(user_id=182e7de0d85f40f0941829018ce80365)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:47.910074 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3366c37f-c416-481a-9d8a-4d9d7bcfb2b6 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:47.911733 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 137/290] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:06:47 2022] GET /identity/v3/users/182e7de0d85f40f0941829018ce80365 => generated 297 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:47.917959 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3366c37f-c416-481a-9d8a-4d9d7bcfb2b6 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:47.985320 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 154/291] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Aug 5 14:06:47 2022] PUT /identity/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b/users/441cf4b2b234489caab48aa696b8080a/roles/8e4d742669e54e288bac0e45718908b8 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:06:48.039444 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3366c37f-c416-481a-9d8a-4d9d7bcfb2b6 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:48.077009 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fd060d-543b-428f-8c63-38e37048f336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:48.077637 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd060d-543b-428f-8c63-38e37048f336 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:48.077976 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd060d-543b-428f-8c63-38e37048f336 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:48.078333 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd060d-543b-428f-8c63-38e37048f336 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:48.128575 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:48.157438 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fd060d-543b-428f-8c63-38e37048f336 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:48.158829 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fd060d-543b-428f-8c63-38e37048f336 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:48.266610 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 138/292] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:06:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 14:06:48.272306 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c763332-eb50-4bb5-9dc8-05767848a2ad None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:48.362527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:48.379505 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c763332-eb50-4bb5-9dc8-05767848a2ad None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:48.416025 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92fcdb6-bef5-4726-be9f-6a1bd3997f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:48.416618 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92fcdb6-bef5-4726-be9f-6a1bd3997f76 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:48.416897 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92fcdb6-bef5-4726-be9f-6a1bd3997f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:48.417167 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92fcdb6-bef5-4726-be9f-6a1bd3997f76 None admin] PATH_INFO: `/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:48.418333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92fcdb6-bef5-4726-be9f-6a1bd3997f76 None admin] RBAC: Authorizing `identity:get_project(project_id=ecbbd8ae8eed4ce5b441221e8e17663b)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:48.419913 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92fcdb6-bef5-4726-be9f-6a1bd3997f76 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:48.421984 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 155/293] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:06:48 2022] GET /identity/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:48.428806 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fd060d-543b-428f-8c63-38e37048f336 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:48.506632 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:48.534399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fd060d-543b-428f-8c63-38e37048f336 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:48.574270 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009cd5d1-c446-41be-903b-7b121687c51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:48.574884 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cd5d1-c446-41be-903b-7b121687c51b None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:48.575157 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cd5d1-c446-41be-903b-7b121687c51b None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:48.575426 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cd5d1-c446-41be-903b-7b121687c51b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:48.576208 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009cd5d1-c446-41be-903b-7b121687c51b None admin] RBAC: Adding query filter params (role.id=29967df108484a30a4c006cde2ae2940, scope.project.id=ecbbd8ae8eed4ce5b441221e8e17663b, user.id=182e7de0d85f40f0941829018ce80365) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:48.576538 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009cd5d1-c446-41be-903b-7b121687c51b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:48.578011 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009cd5d1-c446-41be-903b-7b121687c51b None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:48.584891 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 139/294] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:06:48 2022] GET /identity/v3/role_assignments?user.id=182e7de0d85f40f0941829018ce80365&scope.project.id=ecbbd8ae8eed4ce5b441221e8e17663b&role.id=29967df108484a30a4c006cde2ae2940 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:49.147598 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c93420-bee7-48f6-86ff-d8e6154ec659 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:49.147973 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c93420-bee7-48f6-86ff-d8e6154ec659 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:49.148149 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c93420-bee7-48f6-86ff-d8e6154ec659 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:49.149920 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 156/295] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:49.155057 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3305edb-ce09-4fe8-9ddd-d159c43e4247 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:49.155528 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3305edb-ce09-4fe8-9ddd-d159c43e4247 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:49.155604 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3305edb-ce09-4fe8-9ddd-d159c43e4247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:49.181238 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-d3305edb-ce09-4fe8-9ddd-d159c43e4247 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:49.182758 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d3305edb-ce09-4fe8-9ddd-d159c43e4247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:49.260479 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 140/296] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:49.277578 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c93420-bee7-48f6-86ff-d8e6154ec659 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:49.388737 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c93420-bee7-48f6-86ff-d8e6154ec659 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:49.392650 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8e217-9639-42f8-9456-34bfaa6573b1 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:49.393151 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8e217-9639-42f8-9456-34bfaa6573b1 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:49.394474 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8e217-9639-42f8-9456-34bfaa6573b1 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:49.396404 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 141/297] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:06:49.406058 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2558af-32fe-4c54-9728-6c95f55f3ce1 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:49.406289 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2558af-32fe-4c54-9728-6c95f55f3ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:49.406538 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2558af-32fe-4c54-9728-6c95f55f3ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:49.432443 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-7a2558af-32fe-4c54-9728-6c95f55f3ce1 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:49.434130 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-7a2558af-32fe-4c54-9728-6c95f55f3ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:49.437444 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a1417-ab34-479b-92f3-b0aba07f5a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:49.438413 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a1417-ab34-479b-92f3-b0aba07f5a37 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:49.438901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a1417-ab34-479b-92f3-b0aba07f5a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:49.439378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a1417-ab34-479b-92f3-b0aba07f5a37 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:49.451854 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a1417-ab34-479b-92f3-b0aba07f5a37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:49.457075 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-699a1417-ab34-479b-92f3-b0aba07f5a37 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 14:06:49.457075 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:49.457075 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:49.457075 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:49.457075 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:49.457075 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:49.457075 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:49.457075 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:49.457075 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:49.457075 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:49.457075 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:49.457075 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:49.457075 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:49.457075 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:49.457075 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:49.457075 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:49.457920 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:49.457920 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:49.457920 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:49.457920 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:49.457920 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:49.457920 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:49.457920 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:49.457920 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:49.457920 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:49.457920 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:49.457920 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:49.457920 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:49.457920 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:49.457920 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:49.457920 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:49.457920 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:49.459499 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:49.459499 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:49.459499 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:49.459499 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:49.459499 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:49.459499 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:49.459499 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:49.459499 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:49.459499 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:49.459499 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:49.459499 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 14:06:49.459499 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:06:49.462445 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 157/298] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:06:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:06:49.469126 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a1417-ab34-479b-92f3-b0aba07f5a37 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:49.490868 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:49.511284 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:49.520539 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 142/299] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 14:06:49.529280 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c1bb0-5700-4218-a5bf-80b6020b37c2 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:49.529787 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c1bb0-5700-4218-a5bf-80b6020b37c2 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:49.530250 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c1bb0-5700-4218-a5bf-80b6020b37c2 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:49.532069 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 143/300] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:06:49.537896 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20b308-a64f-4372-bc2d-44bba9366884 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:49.538178 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20b308-a64f-4372-bc2d-44bba9366884 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:49.538457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20b308-a64f-4372-bc2d-44bba9366884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:49.562317 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-9d20b308-a64f-4372-bc2d-44bba9366884 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:49.563481 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-9d20b308-a64f-4372-bc2d-44bba9366884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:49.587786 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a1417-ab34-479b-92f3-b0aba07f5a37 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:49.623735 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8798ea1f-7d23-41c7-be17-2d15728b860f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:49.624346 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798ea1f-7d23-41c7-be17-2d15728b860f None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:49.624744 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798ea1f-7d23-41c7-be17-2d15728b860f None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:49.625012 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798ea1f-7d23-41c7-be17-2d15728b860f None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:49.625714 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8798ea1f-7d23-41c7-be17-2d15728b860f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:49.626009 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8798ea1f-7d23-41c7-be17-2d15728b860f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:49.627351 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8798ea1f-7d23-41c7-be17-2d15728b860f None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:49.636458 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 144/301] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:49.636458 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20b308-a64f-4372-bc2d-44bba9366884 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:49.638928 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 158/302] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Aug 5 14:06:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:06:49.644681 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8798ea1f-7d23-41c7-be17-2d15728b860f None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:49.734848 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:49.745114 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20b308-a64f-4372-bc2d-44bba9366884 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:49.758877 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8798ea1f-7d23-41c7-be17-2d15728b860f None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:49.772434 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:49.779997 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017d4cf5-126d-4364-8ce6-a288fa994f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:49.780666 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017d4cf5-126d-4364-8ce6-a288fa994f34 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:49.780943 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017d4cf5-126d-4364-8ce6-a288fa994f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:49.781211 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017d4cf5-126d-4364-8ce6-a288fa994f34 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:49.782369 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017d4cf5-126d-4364-8ce6-a288fa994f34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:49.783795 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017d4cf5-126d-4364-8ce6-a288fa994f34 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:49.785401 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 145/303] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:06:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:49.791807 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017d4cf5-126d-4364-8ce6-a288fa994f34 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:49.797374 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea6a5f7-567f-459f-bc06-1c6140589132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:49.798253 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea6a5f7-567f-459f-bc06-1c6140589132 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:49.798708 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea6a5f7-567f-459f-bc06-1c6140589132 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:49.799156 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea6a5f7-567f-459f-bc06-1c6140589132 None admin] PATH_INFO: `/v3/users/c81834b2603448fcb1b6b9e58c5f579e` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:49.812987 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea6a5f7-567f-459f-bc06-1c6140589132 None admin] RBAC: Authorizing `identity:get_user(user_id=c81834b2603448fcb1b6b9e58c5f579e)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:49.814416 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea6a5f7-567f-459f-bc06-1c6140589132 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:49.816337 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 159/304] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:06:49 2022] GET /identity/v3/users/c81834b2603448fcb1b6b9e58c5f579e => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:49.821625 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea6a5f7-567f-459f-bc06-1c6140589132 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:49.896293 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017d4cf5-126d-4364-8ce6-a288fa994f34 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:49.922938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea6a5f7-567f-459f-bc06-1c6140589132 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:49.930534 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c58dc81-3f1d-4fd3-a47d-8b9521d432e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:49.931357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c58dc81-3f1d-4fd3-a47d-8b9521d432e2 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:49.931777 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c58dc81-3f1d-4fd3-a47d-8b9521d432e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:49.932188 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c58dc81-3f1d-4fd3-a47d-8b9521d432e2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:49.942498 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c58dc81-3f1d-4fd3-a47d-8b9521d432e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:49.948898 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-1c58dc81-3f1d-4fd3-a47d-8b9521d432e2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 14:06:49.948898 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:49.948898 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:49.948898 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:49.948898 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:49.948898 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:49.948898 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:49.948898 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:49.948898 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:49.948898 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:49.948898 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:49.948898 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:49.948898 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:49.948898 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:49.948898 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:49.948898 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:49.949993 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:49.949993 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:49.949993 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:49.949993 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:49.949993 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:49.949993 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:49.949993 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:49.949993 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:49.949993 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:49.949993 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:49.949993 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:49.949993 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:49.949993 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:49.949993 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:49.949993 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:49.949993 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:49.950992 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:49.950992 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:49.950992 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:49.950992 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:49.950992 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:49.950992 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:49.950992 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:49.950992 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:49.950992 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:49.950992 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:49.950992 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 14:06:49.950992 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:06:49.958028 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 146/305] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:06:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:06:49.959287 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c58dc81-3f1d-4fd3-a47d-8b9521d432e2 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:49.959405 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca019384-60b4-4647-a9b3-2e292b37c46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:49.960018 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca019384-60b4-4647-a9b3-2e292b37c46b None admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:49.960297 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca019384-60b4-4647-a9b3-2e292b37c46b None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:49.960621 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca019384-60b4-4647-a9b3-2e292b37c46b None admin] PATH_INFO: `/v3/groups` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:49.961281 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca019384-60b4-4647-a9b3-2e292b37c46b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:49.962696 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca019384-60b4-4647-a9b3-2e292b37c46b None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:49.966793 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:50.018170 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:50.058897 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 160/306] 10.222.0.26 () {60 vars in 1134 bytes} [Fri Aug 5 14:06:49 2022] POST /identity/v3/groups => generated 229 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:06:50.059732 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca019384-60b4-4647-a9b3-2e292b37c46b None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:50.074588 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c58dc81-3f1d-4fd3-a47d-8b9521d432e2 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:50.115820 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a570f1-1546-4659-8ebb-8616c04999ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:50.116499 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a570f1-1546-4659-8ebb-8616c04999ab None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:50.116783 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a570f1-1546-4659-8ebb-8616c04999ab None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:50.117109 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a570f1-1546-4659-8ebb-8616c04999ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:50.117934 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a570f1-1546-4659-8ebb-8616c04999ab None admin] RBAC: Adding query filter params (role.id=8e4d742669e54e288bac0e45718908b8, user.id=c81834b2603448fcb1b6b9e58c5f579e, scope.system=all) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:50.118251 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a570f1-1546-4659-8ebb-8616c04999ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:50.119867 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a570f1-1546-4659-8ebb-8616c04999ab None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:50.132456 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 147/307] 10.222.0.26 () {56 vars in 1308 bytes} [Fri Aug 5 14:06:49 2022] GET /identity/v3/role_assignments?user.id=c81834b2603448fcb1b6b9e58c5f579e&scope.system=all&role.id=8e4d742669e54e288bac0e45718908b8 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:50.195316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca019384-60b4-4647-a9b3-2e292b37c46b None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:50.235095 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d7526-dd8a-4a5a-8b29-a9612ea871f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:50.235733 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d7526-dd8a-4a5a-8b29-a9612ea871f3 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:50.236022 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d7526-dd8a-4a5a-8b29-a9612ea871f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:50.236296 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d7526-dd8a-4a5a-8b29-a9612ea871f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:50.237067 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26d7526-dd8a-4a5a-8b29-a9612ea871f3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:50.237370 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26d7526-dd8a-4a5a-8b29-a9612ea871f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:50.238750 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26d7526-dd8a-4a5a-8b29-a9612ea871f3 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:50.248979 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 161/308] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Aug 5 14:06:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:50.256012 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a570f1-1546-4659-8ebb-8616c04999ab None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:50.354872 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d758b-3a75-4c4d-a707-eca3884133d0 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:50.355159 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d758b-3a75-4c4d-a707-eca3884133d0 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:50.355340 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d758b-3a75-4c4d-a707-eca3884133d0 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:50.356974 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 162/309] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:06:50.361843 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76345ef7-2204-4869-9933-543db145e70a None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:50.362121 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76345ef7-2204-4869-9933-543db145e70a None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:50.362396 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76345ef7-2204-4869-9933-543db145e70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:50.371826 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a570f1-1546-4659-8ebb-8616c04999ab None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:50.385304 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-76345ef7-2204-4869-9933-543db145e70a None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:50.386511 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-76345ef7-2204-4869-9933-543db145e70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:50.397593 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:50.420663 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca4683-87d0-43b5-9902-bb29e7982f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:50.421320 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca4683-87d0-43b5-9902-bb29e7982f8f None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:50.425893 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca4683-87d0-43b5-9902-bb29e7982f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:50.425893 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca4683-87d0-43b5-9902-bb29e7982f8f None admin] PATH_INFO: `/v3/users/1863a3bc116f4814a3e10925dbfba498` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:50.425893 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adca4683-87d0-43b5-9902-bb29e7982f8f None admin] RBAC: Authorizing `identity:get_user(user_id=1863a3bc116f4814a3e10925dbfba498)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:50.426153 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adca4683-87d0-43b5-9902-bb29e7982f8f None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:50.428123 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 148/310] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:06:50 2022] GET /identity/v3/users/1863a3bc116f4814a3e10925dbfba498 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:50.433792 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca4683-87d0-43b5-9902-bb29e7982f8f None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:50.465619 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:50.474948 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 163/311] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 14:06:50.494147 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76345ef7-2204-4869-9933-543db145e70a None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:50.562824 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca4683-87d0-43b5-9902-bb29e7982f8f None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:50.601280 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704932d-9755-494c-980c-9312c7f0ea3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:50.602261 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704932d-9755-494c-980c-9312c7f0ea3f None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:50.602701 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704932d-9755-494c-980c-9312c7f0ea3f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:50.603112 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704932d-9755-494c-980c-9312c7f0ea3f None admin] PATH_INFO: `/v3/system/users/1863a3bc116f4814a3e10925dbfba498/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:50.604848 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9704932d-9755-494c-980c-9312c7f0ea3f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1863a3bc116f4814a3e10925dbfba498, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:50.606489 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9704932d-9755-494c-980c-9312c7f0ea3f None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:50.630022 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76345ef7-2204-4869-9933-543db145e70a None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:50.667781 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd71dd-865b-427d-a83f-17b00210602a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:50.668446 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd71dd-865b-427d-a83f-17b00210602a None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:50.668729 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd71dd-865b-427d-a83f-17b00210602a None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:50.668998 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd71dd-865b-427d-a83f-17b00210602a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:50.680008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dd71dd-865b-427d-a83f-17b00210602a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:50.685310 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-22dd71dd-865b-427d-a83f-17b00210602a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 14:06:50.685310 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:50.685310 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:50.685310 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:50.685310 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:50.685310 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:50.685310 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:50.685310 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:50.685310 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:50.685310 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:50.685310 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:50.685310 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:50.685310 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:50.685310 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:50.685310 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:50.685310 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:50.690586 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:50.690586 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:50.690586 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:50.690586 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:50.690586 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:50.690586 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:50.690586 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:50.690586 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:50.690586 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:50.690586 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:50.690586 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:50.690586 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:50.690586 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:50.690586 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:50.690586 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:50.690586 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:50.691433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:50.691433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:50.691433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:50.691433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:50.691433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:50.691433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:50.691433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:50.691433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:50.691433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:50.691433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:50.691433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 14:06:50.691433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:06:50.691433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 164/312] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:06:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:06:50.695723 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd71dd-865b-427d-a83f-17b00210602a None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:50.707105 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 149/313] 10.222.0.26 () {58 vars in 1334 bytes} [Fri Aug 5 14:06:50 2022] PUT /identity/v3/system/users/1863a3bc116f4814a3e10925dbfba498/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:06:50.760735 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:50.824562 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd71dd-865b-427d-a83f-17b00210602a None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:50.864467 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9b9d87-77e0-4949-a5c7-82d9b9083e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:50.866179 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b9d87-77e0-4949-a5c7-82d9b9083e2c None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:50.866604 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b9d87-77e0-4949-a5c7-82d9b9083e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:50.867013 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b9d87-77e0-4949-a5c7-82d9b9083e2c None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:50.867867 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9b9d87-77e0-4949-a5c7-82d9b9083e2c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:50.868305 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9b9d87-77e0-4949-a5c7-82d9b9083e2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:50.869843 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9b9d87-77e0-4949-a5c7-82d9b9083e2c None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:50.879258 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 165/314] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Aug 5 14:06:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:50.886628 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704932d-9755-494c-980c-9312c7f0ea3f None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:50.949712 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:50.999460 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704932d-9755-494c-980c-9312c7f0ea3f None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:51.035046 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b5dea-b1ee-4b56-8b95-422067688068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:51.036071 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b5dea-b1ee-4b56-8b95-422067688068 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:51.036547 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b5dea-b1ee-4b56-8b95-422067688068 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:51.037092 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b5dea-b1ee-4b56-8b95-422067688068 None admin] PATH_INFO: `/v3/users/441cf4b2b234489caab48aa696b8080a` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:51.038439 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4b5dea-b1ee-4b56-8b95-422067688068 None admin] RBAC: Authorizing `identity:get_user(user_id=441cf4b2b234489caab48aa696b8080a)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:51.040045 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4b5dea-b1ee-4b56-8b95-422067688068 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:51.041856 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 150/315] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:06:50 2022] GET /identity/v3/users/441cf4b2b234489caab48aa696b8080a => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:51.050451 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9b9d87-77e0-4949-a5c7-82d9b9083e2c None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:51.154988 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9b9d87-77e0-4949-a5c7-82d9b9083e2c None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:51.165286 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:51.196668 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c98711-4621-4d36-b252-7c510d72e1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:51.201094 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c98711-4621-4d36-b252-7c510d72e1a5 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:51.201094 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c98711-4621-4d36-b252-7c510d72e1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:51.201094 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c98711-4621-4d36-b252-7c510d72e1a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:51.274011 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c98711-4621-4d36-b252-7c510d72e1a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:51.275699 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c98711-4621-4d36-b252-7c510d72e1a5 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:51.388078 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 166/316] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:06:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 14:06:51.389658 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b5dea-b1ee-4b56-8b95-422067688068 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:51.494168 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b5dea-b1ee-4b56-8b95-422067688068 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:51.515014 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:51.533381 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c173b-6bbe-4da2-9e9e-38a6b6f14f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:51.538334 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c173b-6bbe-4da2-9e9e-38a6b6f14f55 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:51.538334 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c173b-6bbe-4da2-9e9e-38a6b6f14f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:51.538334 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c173b-6bbe-4da2-9e9e-38a6b6f14f55 None admin] PATH_INFO: `/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:51.538334 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93c173b-6bbe-4da2-9e9e-38a6b6f14f55 None admin] RBAC: Authorizing `identity:get_project(project_id=ecbbd8ae8eed4ce5b441221e8e17663b)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:51.538334 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93c173b-6bbe-4da2-9e9e-38a6b6f14f55 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:51.540603 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 151/317] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:06:51 2022] GET /identity/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:51.548433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c98711-4621-4d36-b252-7c510d72e1a5 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:51.556810 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:51.650536 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c98711-4621-4d36-b252-7c510d72e1a5 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:51.687268 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61091fc2-6ddd-4de2-8be3-2414f8afd49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:51.687888 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61091fc2-6ddd-4de2-8be3-2414f8afd49d None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:51.688160 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61091fc2-6ddd-4de2-8be3-2414f8afd49d None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:51.688450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61091fc2-6ddd-4de2-8be3-2414f8afd49d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:51.689237 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61091fc2-6ddd-4de2-8be3-2414f8afd49d None admin] RBAC: Adding query filter params (role.id=8e4d742669e54e288bac0e45718908b8, scope.project.id=ecbbd8ae8eed4ce5b441221e8e17663b, user.id=441cf4b2b234489caab48aa696b8080a) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:51.689547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61091fc2-6ddd-4de2-8be3-2414f8afd49d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:51.690964 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61091fc2-6ddd-4de2-8be3-2414f8afd49d None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:51.704556 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 167/318] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:06:51 2022] GET /identity/v3/role_assignments?user.id=441cf4b2b234489caab48aa696b8080a&scope.project.id=ecbbd8ae8eed4ce5b441221e8e17663b&role.id=8e4d742669e54e288bac0e45718908b8 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:52.180655 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aba080-4c37-4dec-b9dc-34e1dfbfad66 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:52.181025 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aba080-4c37-4dec-b9dc-34e1dfbfad66 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:52.181219 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aba080-4c37-4dec-b9dc-34e1dfbfad66 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:52.182711 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 152/319] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:52.187628 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d4bfc-021c-42ce-8cc1-315c8a9ecea5 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:52.187923 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d4bfc-021c-42ce-8cc1-315c8a9ecea5 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:52.188203 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d4bfc-021c-42ce-8cc1-315c8a9ecea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:52.206656 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-799d4bfc-021c-42ce-8cc1-315c8a9ecea5 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:52.207842 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-799d4bfc-021c-42ce-8cc1-315c8a9ecea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:52.253653 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a9002-3ca6-490a-b805-3852b9dacf64 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:52.254225 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a9002-3ca6-490a-b805-3852b9dacf64 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:52.254506 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a9002-3ca6-490a-b805-3852b9dacf64 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:52.256781 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 153/320] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:52.261753 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8988b0-0523-40d0-b542-a1e19a5fdfe7 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:52.262165 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8988b0-0523-40d0-b542-a1e19a5fdfe7 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:52.262445 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8988b0-0523-40d0-b542-a1e19a5fdfe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:52.278695 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 168/321] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:52.284939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-ae8988b0-0523-40d0-b542-a1e19a5fdfe7 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:52.286130 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ae8988b0-0523-40d0-b542-a1e19a5fdfe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:52.300839 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d4bfc-021c-42ce-8cc1-315c8a9ecea5 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:52.303159 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:52.363356 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 154/322] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 14:06:52.385995 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8988b0-0523-40d0-b542-a1e19a5fdfe7 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:52.412630 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d4bfc-021c-42ce-8cc1-315c8a9ecea5 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:52.451113 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84577d08-8b9b-41b1-a27a-1059158e8a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:52.452001 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84577d08-8b9b-41b1-a27a-1059158e8a43 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:52.452279 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84577d08-8b9b-41b1-a27a-1059158e8a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:52.452674 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84577d08-8b9b-41b1-a27a-1059158e8a43 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:52.453828 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84577d08-8b9b-41b1-a27a-1059158e8a43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:52.455271 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84577d08-8b9b-41b1-a27a-1059158e8a43 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:52.457086 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 169/323] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:06:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:06:52.462432 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84577d08-8b9b-41b1-a27a-1059158e8a43 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:52.492208 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8988b0-0523-40d0-b542-a1e19a5fdfe7 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:52.510531 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:52.538375 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc187c-5888-4054-a80f-dece710a6e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:52.538375 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc187c-5888-4054-a80f-dece710a6e7f None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:52.538375 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc187c-5888-4054-a80f-dece710a6e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:52.538375 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc187c-5888-4054-a80f-dece710a6e7f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:52.543219 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bc187c-5888-4054-a80f-dece710a6e7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:52.545430 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:52.553357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-65bc187c-5888-4054-a80f-dece710a6e7f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 14:06:52.553357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:52.553357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:52.553357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:52.553357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:52.553357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:52.553357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:52.553357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:52.553357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:52.553357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:52.553357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:52.553357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:52.553357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:52.553357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:52.553357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:52.553357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:52.554403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:52.554403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:52.554403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:52.554403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:52.554403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:52.554403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:52.554403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:52.554403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:52.554403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:52.554403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:52.554403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:52.554403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:52.554403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:52.554403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:52.554403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:52.554403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:52.555556 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:52.555556 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:52.555556 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:52.555556 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:52.555556 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:52.555556 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:52.555556 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:52.555556 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:52.555556 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:52.555556 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:52.555556 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 14:06:52.555556 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:06:52.560788 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 155/324] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:06:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:06:52.562221 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc187c-5888-4054-a80f-dece710a6e7f None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:52.580691 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84577d08-8b9b-41b1-a27a-1059158e8a43 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:52.618291 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3aa58b-31f4-454b-b22a-b9641c19fe22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:52.618944 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3aa58b-31f4-454b-b22a-b9641c19fe22 None admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:52.619367 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3aa58b-31f4-454b-b22a-b9641c19fe22 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:52.619646 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3aa58b-31f4-454b-b22a-b9641c19fe22 None admin] PATH_INFO: `/v3/groups` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:52.620310 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3aa58b-31f4-454b-b22a-b9641c19fe22 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:52.621784 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3aa58b-31f4-454b-b22a-b9641c19fe22 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:52.669534 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc187c-5888-4054-a80f-dece710a6e7f None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:52.695704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 170/325] 10.222.0.26 () {60 vars in 1134 bytes} [Fri Aug 5 14:06:52 2022] POST /identity/v3/groups => generated 226 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:06:52.706849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ef29c-9b59-402f-9ad7-83ee7e5fa003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:52.707426 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ef29c-9b59-402f-9ad7-83ee7e5fa003 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:52.707705 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ef29c-9b59-402f-9ad7-83ee7e5fa003 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:52.707974 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ef29c-9b59-402f-9ad7-83ee7e5fa003 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:52.709300 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764ef29c-9b59-402f-9ad7-83ee7e5fa003 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:52.709606 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764ef29c-9b59-402f-9ad7-83ee7e5fa003 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:52.711138 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764ef29c-9b59-402f-9ad7-83ee7e5fa003 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:52.721140 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 156/326] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Aug 5 14:06:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:06:52.727335 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3aa58b-31f4-454b-b22a-b9641c19fe22 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:52.822296 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24863e38-a32e-425a-b6b2-306eb3aae6b8 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:52.822533 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24863e38-a32e-425a-b6b2-306eb3aae6b8 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:52.822758 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24863e38-a32e-425a-b6b2-306eb3aae6b8 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:52.824499 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 157/327] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:06:52.829667 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb73569-543c-4f03-9b89-71adfce25e42 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:52.829958 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb73569-543c-4f03-9b89-71adfce25e42 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:52.830227 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb73569-543c-4f03-9b89-71adfce25e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:52.841596 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3aa58b-31f4-454b-b22a-b9641c19fe22 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:52.852810 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-fdb73569-543c-4f03-9b89-71adfce25e42 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:52.854331 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-fdb73569-543c-4f03-9b89-71adfce25e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:52.873674 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:52.884367 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e48441-c56f-4649-b101-86ef2e6849cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:52.885088 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e48441-c56f-4649-b101-86ef2e6849cf None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:52.885331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e48441-c56f-4649-b101-86ef2e6849cf None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:52.885610 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e48441-c56f-4649-b101-86ef2e6849cf None admin] PATH_INFO: `/v3/users/c81834b2603448fcb1b6b9e58c5f579e` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:52.888087 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e48441-c56f-4649-b101-86ef2e6849cf None admin] RBAC: Authorizing `identity:get_user(user_id=c81834b2603448fcb1b6b9e58c5f579e)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:52.888291 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e48441-c56f-4649-b101-86ef2e6849cf None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:52.890046 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 171/328] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:06:52 2022] GET /identity/v3/users/c81834b2603448fcb1b6b9e58c5f579e => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:52.895471 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e48441-c56f-4649-b101-86ef2e6849cf None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:52.899270 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:52.941880 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 158/329] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 14:06:52.967444 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb73569-543c-4f03-9b89-71adfce25e42 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:53.026893 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e48441-c56f-4649-b101-86ef2e6849cf None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:53.070474 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e593f-196a-4dd1-95d2-da28cdf3ff28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:53.071143 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e593f-196a-4dd1-95d2-da28cdf3ff28 None admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:53.071419 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e593f-196a-4dd1-95d2-da28cdf3ff28 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:53.071692 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e593f-196a-4dd1-95d2-da28cdf3ff28 None admin] PATH_INFO: `/v3/system/users/c81834b2603448fcb1b6b9e58c5f579e/roles/8e4d742669e54e288bac0e45718908b8` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:53.074184 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713e593f-196a-4dd1-95d2-da28cdf3ff28 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c81834b2603448fcb1b6b9e58c5f579e, role_id=8e4d742669e54e288bac0e45718908b8)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:53.075595 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713e593f-196a-4dd1-95d2-da28cdf3ff28 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:53.088893 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb73569-543c-4f03-9b89-71adfce25e42 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:53.126630 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba874801-2098-4322-bfdd-82d43ba419e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:53.127281 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba874801-2098-4322-bfdd-82d43ba419e9 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:53.127765 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba874801-2098-4322-bfdd-82d43ba419e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:53.128050 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba874801-2098-4322-bfdd-82d43ba419e9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:53.139919 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba874801-2098-4322-bfdd-82d43ba419e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:53.145146 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-ba874801-2098-4322-bfdd-82d43ba419e9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 14:06:53.145146 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:53.145146 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:53.145146 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:53.145146 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:53.145146 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:53.145146 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:53.145146 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:53.145146 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:53.145146 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:53.145146 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:53.145146 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:53.145146 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:53.145146 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:53.145146 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:53.145146 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:53.146774 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:53.146774 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:53.146774 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:53.146774 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:53.146774 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:53.146774 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:53.146774 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:53.146774 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:53.146774 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:53.146774 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:53.146774 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:53.146774 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:53.146774 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:53.146774 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:53.146774 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:53.146774 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:53.147697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:53.147697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:53.147697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:53.147697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:53.147697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:53.147697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:53.147697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:53.147697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:53.147697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:53.147697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:53.147697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 14:06:53.147697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:06:53.147697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 159/330] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:06:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:06:53.150857 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 172/331] 10.222.0.26 () {58 vars in 1334 bytes} [Fri Aug 5 14:06:52 2022] PUT /identity/v3/system/users/c81834b2603448fcb1b6b9e58c5f579e/roles/8e4d742669e54e288bac0e45718908b8 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:06:53.153500 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba874801-2098-4322-bfdd-82d43ba419e9 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:53.189459 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:53.275306 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba874801-2098-4322-bfdd-82d43ba419e9 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:53.317752 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bae181-cc90-4313-afb6-18703b8e3669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:53.319490 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bae181-cc90-4313-afb6-18703b8e3669 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:53.319978 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bae181-cc90-4313-afb6-18703b8e3669 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:53.320472 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bae181-cc90-4313-afb6-18703b8e3669 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:53.321434 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bae181-cc90-4313-afb6-18703b8e3669 None admin] RBAC: Adding query filter params (name=member) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:53.322039 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bae181-cc90-4313-afb6-18703b8e3669 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:53.323647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bae181-cc90-4313-afb6-18703b8e3669 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:53.334413 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 160/332] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Aug 5 14:06:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:53.339974 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e593f-196a-4dd1-95d2-da28cdf3ff28 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:53.414779 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:53.459157 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e593f-196a-4dd1-95d2-da28cdf3ff28 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:53.496471 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de4826a-03d8-4abe-834e-7c4f47134403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:53.497365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4826a-03d8-4abe-834e-7c4f47134403 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:53.498060 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4826a-03d8-4abe-834e-7c4f47134403 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:53.498333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4826a-03d8-4abe-834e-7c4f47134403 None admin] PATH_INFO: `/v3/users/1863a3bc116f4814a3e10925dbfba498` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:53.499515 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de4826a-03d8-4abe-834e-7c4f47134403 None admin] RBAC: Authorizing `identity:get_user(user_id=1863a3bc116f4814a3e10925dbfba498)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:53.500983 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de4826a-03d8-4abe-834e-7c4f47134403 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:53.502891 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 173/333] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:06:53 2022] GET /identity/v3/users/1863a3bc116f4814a3e10925dbfba498 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:06:53.509270 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bae181-cc90-4313-afb6-18703b8e3669 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:53.619487 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:53.629385 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bae181-cc90-4313-afb6-18703b8e3669 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:53.665972 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c926a384-fb0c-4926-8c56-78027a731df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:53.666590 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926a384-fb0c-4926-8c56-78027a731df9 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:53.666866 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926a384-fb0c-4926-8c56-78027a731df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:53.667134 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926a384-fb0c-4926-8c56-78027a731df9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:53.667921 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c926a384-fb0c-4926-8c56-78027a731df9 None admin] RBAC: Adding query filter params (role.id=29967df108484a30a4c006cde2ae2940, user.id=1863a3bc116f4814a3e10925dbfba498, scope.system=all) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:53.668227 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c926a384-fb0c-4926-8c56-78027a731df9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:53.669701 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c926a384-fb0c-4926-8c56-78027a731df9 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:53.687433 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 161/334] 10.222.0.26 () {56 vars in 1308 bytes} [Fri Aug 5 14:06:53 2022] GET /identity/v3/role_assignments?user.id=1863a3bc116f4814a3e10925dbfba498&scope.system=all&role.id=29967df108484a30a4c006cde2ae2940 => generated 510 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:06:55.060703 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c4560-5210-413e-a5b3-1ab555f3940e None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:55.061763 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c4560-5210-413e-a5b3-1ab555f3940e None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:55.062247 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c4560-5210-413e-a5b3-1ab555f3940e None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:55.064127 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 174/335] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:55.070055 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddf7c9-9f0e-4d05-873f-f86b1742e708 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:55.070940 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddf7c9-9f0e-4d05-873f-f86b1742e708 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:55.071422 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddf7c9-9f0e-4d05-873f-f86b1742e708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:55.100369 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-15ddf7c9-9f0e-4d05-873f-f86b1742e708 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:55.101926 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-15ddf7c9-9f0e-4d05-873f-f86b1742e708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:55.182064 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 162/336] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:55.217523 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c4560-5210-413e-a5b3-1ab555f3940e None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:55.331210 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c4560-5210-413e-a5b3-1ab555f3940e None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:55.364578 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:55.371705 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d5cdc-d13c-48f1-b602-4d172ae4ba09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:55.372685 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d5cdc-d13c-48f1-b602-4d172ae4ba09 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:55.372965 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d5cdc-d13c-48f1-b602-4d172ae4ba09 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:55.373325 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d5cdc-d13c-48f1-b602-4d172ae4ba09 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:55.389146 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639d5cdc-d13c-48f1-b602-4d172ae4ba09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:55.396675 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-639d5cdc-d13c-48f1-b602-4d172ae4ba09 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 14:06:55.396675 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:55.396675 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:55.396675 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:55.396675 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:55.396675 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:55.396675 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:55.396675 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:55.396675 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:55.396675 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:55.396675 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:55.396675 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:55.396675 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:55.396675 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:55.396675 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:55.396675 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:55.397745 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:55.397745 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:55.397745 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:55.397745 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:55.397745 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:55.397745 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:55.397745 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:55.397745 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:55.397745 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:55.397745 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:55.397745 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:55.397745 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:55.397745 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:55.397745 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:55.397745 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:55.397745 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:55.398528 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:55.398528 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:55.398528 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:55.398528 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:55.398528 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:55.398528 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:55.398528 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:55.398528 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:55.398528 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:55.398528 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:55.398528 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 14:06:55.398528 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:06:55.398528 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 175/337] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:06:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:06:55.404128 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ddf7c9-9f0e-4d05-873f-f86b1742e708 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:55.506076 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137d521-fd39-44f4-8554-22c5b56fe840 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:55.506707 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137d521-fd39-44f4-8554-22c5b56fe840 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:55.507201 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137d521-fd39-44f4-8554-22c5b56fe840 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:55.507677 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:55.509307 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 176/338] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:55.523268 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916fea8-42b0-422d-9aff-b40bc50d30cd None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:55.523871 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916fea8-42b0-422d-9aff-b40bc50d30cd None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:55.524348 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916fea8-42b0-422d-9aff-b40bc50d30cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:55.540141 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ddf7c9-9f0e-4d05-873f-f86b1742e708 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:55.559553 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-1916fea8-42b0-422d-9aff-b40bc50d30cd None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:55.560804 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-1916fea8-42b0-422d-9aff-b40bc50d30cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:55.604912 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770172b-ac72-4f09-9bbc-75553de045e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:55.605885 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770172b-ac72-4f09-9bbc-75553de045e3 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:55.606380 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770172b-ac72-4f09-9bbc-75553de045e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:55.606846 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770172b-ac72-4f09-9bbc-75553de045e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:55.607770 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8770172b-ac72-4f09-9bbc-75553de045e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:55.608269 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8770172b-ac72-4f09-9bbc-75553de045e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:55.609977 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8770172b-ac72-4f09-9bbc-75553de045e3 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:55.627581 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 163/339] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Aug 5 14:06:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:55.633743 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770172b-ac72-4f09-9bbc-75553de045e3 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:55.651560 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 177/340] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:55.677071 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1916fea8-42b0-422d-9aff-b40bc50d30cd None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:55.784942 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1916fea8-42b0-422d-9aff-b40bc50d30cd None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:55.807202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:55.816354 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770172b-ac72-4f09-9bbc-75553de045e3 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:55.823126 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc331eb-525a-4327-ac5a-8a25e029b597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:55.824072 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc331eb-525a-4327-ac5a-8a25e029b597 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:55.824540 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc331eb-525a-4327-ac5a-8a25e029b597 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:55.825223 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc331eb-525a-4327-ac5a-8a25e029b597 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:55.837033 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc331eb-525a-4327-ac5a-8a25e029b597 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:55.842648 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-ffc331eb-525a-4327-ac5a-8a25e029b597 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 14:06:55.842648 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:55.842648 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:55.842648 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:55.842648 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:55.842648 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:55.842648 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:55.842648 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:55.842648 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:55.842648 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:55.842648 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:55.842648 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:55.842648 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:55.842648 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:55.842648 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:55.842648 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:55.843537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:55.843537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:55.843537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:55.843537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:55.843537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:55.843537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:55.843537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:55.843537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:55.843537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:55.843537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:55.843537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:55.843537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:55.843537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:55.843537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:55.843537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:55.843537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:55.844331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:55.844331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:55.844331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:55.844331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:55.844331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:55.844331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:55.844331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:55.844331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:55.844331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:55.844331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:55.844331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 14:06:55.844331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:06:55.847012 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 178/341] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Aug 5 14:06:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:06:55.854726 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc331eb-525a-4327-ac5a-8a25e029b597 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:55.878434 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6255f29-49bd-41dc-b42a-218d05f0d135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:55.879082 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6255f29-49bd-41dc-b42a-218d05f0d135 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:55.879358 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6255f29-49bd-41dc-b42a-218d05f0d135 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:55.879625 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6255f29-49bd-41dc-b42a-218d05f0d135 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:55.904762 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:55.972279 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6255f29-49bd-41dc-b42a-218d05f0d135 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:55.974572 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6255f29-49bd-41dc-b42a-218d05f0d135 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:55.979002 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc331eb-525a-4327-ac5a-8a25e029b597 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:56.030444 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15416c3f-3a6c-485d-8234-94bcc955e578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:56.031076 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15416c3f-3a6c-485d-8234-94bcc955e578 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:56.031357 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15416c3f-3a6c-485d-8234-94bcc955e578 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:56.031631 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15416c3f-3a6c-485d-8234-94bcc955e578 None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:56.032355 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15416c3f-3a6c-485d-8234-94bcc955e578 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:56.037297 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15416c3f-3a6c-485d-8234-94bcc955e578 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:56.038943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15416c3f-3a6c-485d-8234-94bcc955e578 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:56.053122 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 179/342] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:06:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:56.058860 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15416c3f-3a6c-485d-8234-94bcc955e578 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:56.083295 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 164/343] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:06:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 14:06:56.089008 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6255f29-49bd-41dc-b42a-218d05f0d135 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:56.158654 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:56.184588 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:56.203713 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6255f29-49bd-41dc-b42a-218d05f0d135 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:56.270365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15416c3f-3a6c-485d-8234-94bcc955e578 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:56.272570 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c1140-4de3-4ebd-b5fe-5bb11e3e04bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:56.273466 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c1140-4de3-4ebd-b5fe-5bb11e3e04bf None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:56.273967 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c1140-4de3-4ebd-b5fe-5bb11e3e04bf None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:56.274441 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c1140-4de3-4ebd-b5fe-5bb11e3e04bf None admin] PATH_INFO: `/v3/projects/580a155c67b742b9bc398f50556f26cf` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:56.276631 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5c1140-4de3-4ebd-b5fe-5bb11e3e04bf None admin] RBAC: Authorizing `identity:get_project(project_id=580a155c67b742b9bc398f50556f26cf)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:56.278295 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5c1140-4de3-4ebd-b5fe-5bb11e3e04bf None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:56.284508 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 165/344] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:06:56 2022] GET /identity/v3/projects/580a155c67b742b9bc398f50556f26cf => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:06:56.286033 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71289472-209a-465c-9304-54f936dcd217 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:56.286622 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71289472-209a-465c-9304-54f936dcd217 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:56.287733 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71289472-209a-465c-9304-54f936dcd217 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:56.289375 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 166/345] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:56.294586 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad87544-a8b5-4053-b01c-85c2cc6ff67d None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:56.295099 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad87544-a8b5-4053-b01c-85c2cc6ff67d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:56.295577 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad87544-a8b5-4053-b01c-85c2cc6ff67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:56.315126 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad836aab-56ec-4748-ad01-e406ef93863f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:56.315809 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad836aab-56ec-4748-ad01-e406ef93863f None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:56.316087 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad836aab-56ec-4748-ad01-e406ef93863f None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:56.316352 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad836aab-56ec-4748-ad01-e406ef93863f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:56.331417 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-8ad87544-a8b5-4053-b01c-85c2cc6ff67d None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:56.332973 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8ad87544-a8b5-4053-b01c-85c2cc6ff67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:56.398290 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad836aab-56ec-4748-ad01-e406ef93863f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:56.399759 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad836aab-56ec-4748-ad01-e406ef93863f None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:56.440083 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 167/346] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 14:06:56.445308 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad87544-a8b5-4053-b01c-85c2cc6ff67d None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:56.450639 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:56.511387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 180/347] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:06:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 14:06:56.512770 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a55145-0da6-479d-a3fd-70f8f5ab6e8d None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:56.513055 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a55145-0da6-479d-a3fd-70f8f5ab6e8d None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:56.513331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a55145-0da6-479d-a3fd-70f8f5ab6e8d None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:56.514388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:56.516163 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 181/348] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:56.517083 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a55145-0da6-479d-a3fd-70f8f5ab6e8d None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:56.602089 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad87544-a8b5-4053-b01c-85c2cc6ff67d None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:56.646631 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a4a0b-c801-4fe3-af5a-a30d34d2e449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:56.647741 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a4a0b-c801-4fe3-af5a-a30d34d2e449 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:56.648242 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a4a0b-c801-4fe3-af5a-a30d34d2e449 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:56.648659 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a4a0b-c801-4fe3-af5a-a30d34d2e449 None admin] PATH_INFO: `/v3/groups/674c9fb834714b89a516d5f2cc7d5005` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:56.651519 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a55145-0da6-479d-a3fd-70f8f5ab6e8d None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:56.657191 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a4a0b-c801-4fe3-af5a-a30d34d2e449 None admin] RBAC: Authorizing `identity:get_group(group_id=674c9fb834714b89a516d5f2cc7d5005)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:56.658700 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a4a0b-c801-4fe3-af5a-a30d34d2e449 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:56.660606 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 168/349] 10.222.0.26 () {56 vars in 1180 bytes} [Fri Aug 5 14:06:56 2022] GET /identity/v3/groups/674c9fb834714b89a516d5f2cc7d5005 => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:56.661262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a4a0b-c801-4fe3-af5a-a30d34d2e449 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:56.694327 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7109858f-7889-49a2-836c-91a2a23d43ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:56.694900 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109858f-7889-49a2-836c-91a2a23d43ce None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:56.695179 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109858f-7889-49a2-836c-91a2a23d43ce None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:56.695454 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109858f-7889-49a2-836c-91a2a23d43ce None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:56.706531 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7109858f-7889-49a2-836c-91a2a23d43ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:56.712393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-7109858f-7889-49a2-836c-91a2a23d43ce None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 14:06:56.712393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:56.712393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:56.712393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:56.712393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:56.712393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:56.712393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:56.712393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:56.712393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:56.712393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:56.712393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:56.712393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:56.712393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:56.712393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:56.712393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:56.712393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:56.713333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:56.713333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:56.713333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:56.713333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:56.713333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:56.713333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:56.713333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:56.713333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:56.713333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:56.713333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:56.713333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:56.713333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:56.713333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:56.713333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:56.713333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:56.713333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:56.714148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:56.714148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:56.714148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:56.714148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:56.714148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:56.714148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:56.714148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:56.714148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:56.714148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:56.714148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:56.714148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 14:06:56.714148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:06:56.714148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 182/350] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:06:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:06:56.714805 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79651f-b4d1-4f10-83fc-ddb4233b85e3 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:56.715592 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79651f-b4d1-4f10-83fc-ddb4233b85e3 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:56.715592 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79651f-b4d1-4f10-83fc-ddb4233b85e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:56.728309 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:56.740968 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-7e79651f-b4d1-4f10-83fc-ddb4233b85e3 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:56.742211 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-7e79651f-b4d1-4f10-83fc-ddb4233b85e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:56.781033 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a4a0b-c801-4fe3-af5a-a30d34d2e449 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:56.783548 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:56.819361 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 183/351] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:56.819848 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6342f25-462d-484d-9746-c930fdcbfe13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:56.820210 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e79651f-b4d1-4f10-83fc-ddb4233b85e3 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:56.820836 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6342f25-462d-484d-9746-c930fdcbfe13 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:56.821362 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6342f25-462d-484d-9746-c930fdcbfe13 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:56.821810 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6342f25-462d-484d-9746-c930fdcbfe13 None admin] PATH_INFO: `/v3/projects/580a155c67b742b9bc398f50556f26cf` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:56.823220 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6342f25-462d-484d-9746-c930fdcbfe13 None admin] RBAC: Authorizing `identity:get_project(project_id=580a155c67b742b9bc398f50556f26cf)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:56.824978 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6342f25-462d-484d-9746-c930fdcbfe13 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:56.827080 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 169/352] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:06:56 2022] GET /identity/v3/projects/580a155c67b742b9bc398f50556f26cf => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:56.828141 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6342f25-462d-484d-9746-c930fdcbfe13 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:56.944983 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6342f25-462d-484d-9746-c930fdcbfe13 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:56.949809 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e79651f-b4d1-4f10-83fc-ddb4233b85e3 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:56.981528 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:56.984571 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b394b-e943-4b43-bb4b-7fda0b2c636c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:56.985584 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b394b-e943-4b43-bb4b-7fda0b2c636c None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:56.990960 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b394b-e943-4b43-bb4b-7fda0b2c636c None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:56.990960 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b394b-e943-4b43-bb4b-7fda0b2c636c None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:56.991124 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5140c-a28d-4283-a0e7-2618e2d671b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:56.991209 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167b394b-e943-4b43-bb4b-7fda0b2c636c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:56.991209 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167b394b-e943-4b43-bb4b-7fda0b2c636c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:56.991353 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5140c-a28d-4283-a0e7-2618e2d671b4 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:56.991353 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5140c-a28d-4283-a0e7-2618e2d671b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:56.991353 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5140c-a28d-4283-a0e7-2618e2d671b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:56.991522 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167b394b-e943-4b43-bb4b-7fda0b2c636c None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:56.991587 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5140c-a28d-4283-a0e7-2618e2d671b4 None admin] RBAC: Adding query filter params (group.id=674c9fb834714b89a516d5f2cc7d5005, role.id=29967df108484a30a4c006cde2ae2940, scope.project.id=580a155c67b742b9bc398f50556f26cf) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:56.991587 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5140c-a28d-4283-a0e7-2618e2d671b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:56.991587 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5140c-a28d-4283-a0e7-2618e2d671b4 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:56.998387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 184/353] 10.222.0.26 () {56 vars in 1376 bytes} [Fri Aug 5 14:06:56 2022] GET /identity/v3/role_assignments?group.id=674c9fb834714b89a516d5f2cc7d5005&scope.project.id=580a155c67b742b9bc398f50556f26cf&role.id=29967df108484a30a4c006cde2ae2940 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:56.999022 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5140c-a28d-4283-a0e7-2618e2d671b4 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:57.000844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 170/354] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Aug 5 14:06:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:57.001956 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b394b-e943-4b43-bb4b-7fda0b2c636c None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:57.042449 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:57.113379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5140c-a28d-4283-a0e7-2618e2d671b4 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:57.120565 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b394b-e943-4b43-bb4b-7fda0b2c636c None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:57.150846 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c4b85-5a63-44e2-87e9-fff4b163c335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:57.151497 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c4b85-5a63-44e2-87e9-fff4b163c335 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:57.151776 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c4b85-5a63-44e2-87e9-fff4b163c335 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:57.152050 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c4b85-5a63-44e2-87e9-fff4b163c335 None admin] PATH_INFO: `/v3/groups/674c9fb834714b89a516d5f2cc7d5005` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:57.153250 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427c4b85-5a63-44e2-87e9-fff4b163c335 None admin] RBAC: Authorizing `identity:get_group(group_id=674c9fb834714b89a516d5f2cc7d5005)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:57.154784 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427c4b85-5a63-44e2-87e9-fff4b163c335 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:57.156501 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 185/355] 10.222.0.26 () {56 vars in 1180 bytes} [Fri Aug 5 14:06:56 2022] GET /identity/v3/groups/674c9fb834714b89a516d5f2cc7d5005 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:57.157301 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c4b85-5a63-44e2-87e9-fff4b163c335 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:57.164866 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf5fd7-9481-4f6c-9f6f-69319ed75ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:57.165744 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf5fd7-9481-4f6c-9f6f-69319ed75ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:57.166165 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf5fd7-9481-4f6c-9f6f-69319ed75ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:57.166654 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf5fd7-9481-4f6c-9f6f-69319ed75ef1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:57.179978 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cf5fd7-9481-4f6c-9f6f-69319ed75ef1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:57.185314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-20cf5fd7-9481-4f6c-9f6f-69319ed75ef1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 14:06:57.185314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:57.185314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:57.185314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:57.185314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:57.185314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:57.185314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:57.185314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:57.185314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:57.185314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:57.185314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:57.185314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:57.185314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:57.185314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:57.185314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:57.185314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:57.186410 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:57.186410 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:57.186410 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:57.186410 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:57.186410 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:57.186410 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:57.186410 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:57.186410 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:57.186410 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:57.186410 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:57.186410 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:57.186410 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:57.186410 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:57.186410 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:57.186410 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:57.186410 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:57.187387 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:57.187387 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:57.187387 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:57.187387 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:57.187387 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:57.187387 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:57.187387 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:57.187387 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:57.187387 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:57.187387 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:57.187387 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 14:06:57.187387 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:06:57.189702 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 171/356] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:06:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:06:57.190680 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf5fd7-9481-4f6c-9f6f-69319ed75ef1 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:57.228937 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:57.273377 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c4b85-5a63-44e2-87e9-fff4b163c335 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:57.304365 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:57.312664 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf5fd7-9481-4f6c-9f6f-69319ed75ef1 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:57.312856 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f9e2d-6ba3-41c2-b4f0-3401092587ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:57.313511 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f9e2d-6ba3-41c2-b4f0-3401092587ea None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:57.313850 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f9e2d-6ba3-41c2-b4f0-3401092587ea None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:57.314183 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f9e2d-6ba3-41c2-b4f0-3401092587ea None admin] PATH_INFO: `/v3/users/c81834b2603448fcb1b6b9e58c5f579e` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:57.315465 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f9e2d-6ba3-41c2-b4f0-3401092587ea None admin] RBAC: Authorizing `identity:get_user(user_id=c81834b2603448fcb1b6b9e58c5f579e)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:57.317172 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f9e2d-6ba3-41c2-b4f0-3401092587ea None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:57.319530 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 186/357] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:06:57 2022] GET /identity/v3/users/c81834b2603448fcb1b6b9e58c5f579e => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:57.320392 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f9e2d-6ba3-41c2-b4f0-3401092587ea None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:57.371086 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910ee1b-8c09-4c75-8e7f-bf6002fba56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:57.371786 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910ee1b-8c09-4c75-8e7f-bf6002fba56a None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:57.372109 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910ee1b-8c09-4c75-8e7f-bf6002fba56a None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:57.372779 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910ee1b-8c09-4c75-8e7f-bf6002fba56a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:57.373753 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8910ee1b-8c09-4c75-8e7f-bf6002fba56a None admin] RBAC: Adding query filter params (group.id=674c9fb834714b89a516d5f2cc7d5005, role.id=199fd3093a824ba585b1135c8bf363be, scope.project.id=580a155c67b742b9bc398f50556f26cf) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:57.374135 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8910ee1b-8c09-4c75-8e7f-bf6002fba56a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:57.375778 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8910ee1b-8c09-4c75-8e7f-bf6002fba56a None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:57.384197 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 172/358] 10.222.0.26 () {56 vars in 1376 bytes} [Fri Aug 5 14:06:57 2022] GET /identity/v3/role_assignments?group.id=674c9fb834714b89a516d5f2cc7d5005&scope.project.id=580a155c67b742b9bc398f50556f26cf&role.id=199fd3093a824ba585b1135c8bf363be => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:57.385078 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910ee1b-8c09-4c75-8e7f-bf6002fba56a None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:57.432057 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f9e2d-6ba3-41c2-b4f0-3401092587ea None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:57.469071 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce16eaa-d685-41ea-a0ef-0c5b3b1d2ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:57.469995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:57.470491 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce16eaa-d685-41ea-a0ef-0c5b3b1d2ffc None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:57.470952 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce16eaa-d685-41ea-a0ef-0c5b3b1d2ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:57.471398 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce16eaa-d685-41ea-a0ef-0c5b3b1d2ffc None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:57.472295 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce16eaa-d685-41ea-a0ef-0c5b3b1d2ffc None admin] RBAC: Adding query filter params (name=member) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:57.472795 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce16eaa-d685-41ea-a0ef-0c5b3b1d2ffc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:57.474442 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce16eaa-d685-41ea-a0ef-0c5b3b1d2ffc None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:57.486851 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 187/359] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Aug 5 14:06:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:57.488432 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3128e1-596d-4fe7-8080-d8f4443ad5e5 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:57.488954 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3128e1-596d-4fe7-8080-d8f4443ad5e5 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:57.489399 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3128e1-596d-4fe7-8080-d8f4443ad5e5 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:57.490923 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 188/360] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:57.496776 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71300527-60b3-4604-96ab-8f7b3979604a None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:57.497531 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71300527-60b3-4604-96ab-8f7b3979604a None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:57.497979 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71300527-60b3-4604-96ab-8f7b3979604a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:57.518561 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-71300527-60b3-4604-96ab-8f7b3979604a None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:57.521673 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-71300527-60b3-4604-96ab-8f7b3979604a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:57.541901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910ee1b-8c09-4c75-8e7f-bf6002fba56a None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:57.590005 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03592045-21ee-47a1-997a-b4be24e32b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:57.590743 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03592045-21ee-47a1-997a-b4be24e32b52 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:57.591029 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03592045-21ee-47a1-997a-b4be24e32b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:57.591308 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03592045-21ee-47a1-997a-b4be24e32b52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:57.592215 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03592045-21ee-47a1-997a-b4be24e32b52 None admin] RBAC: Adding query filter params (role.id=8e4d742669e54e288bac0e45718908b8, user.id=c81834b2603448fcb1b6b9e58c5f579e, scope.system=all) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:57.592575 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03592045-21ee-47a1-997a-b4be24e32b52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:57.594597 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03592045-21ee-47a1-997a-b4be24e32b52 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:57.603668 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 189/361] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:57.607005 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71300527-60b3-4604-96ab-8f7b3979604a None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:57.609196 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 173/362] 10.222.0.26 () {56 vars in 1308 bytes} [Fri Aug 5 14:06:57 2022] GET /identity/v3/role_assignments?user.id=c81834b2603448fcb1b6b9e58c5f579e&scope.system=all&role.id=8e4d742669e54e288bac0e45718908b8 => generated 510 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:57.657997 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03592045-21ee-47a1-997a-b4be24e32b52 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:57.693702 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:57.725564 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71300527-60b3-4604-96ab-8f7b3979604a None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:57.756006 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:57.769216 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ad24a-527b-45f0-a273-768643660bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:57.769910 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ad24a-527b-45f0-a273-768643660bba None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:57.770191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ad24a-527b-45f0-a273-768643660bba None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:57.770464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ad24a-527b-45f0-a273-768643660bba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:57.820617 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03592045-21ee-47a1-997a-b4be24e32b52 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:57.854988 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ad24a-527b-45f0-a273-768643660bba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:57.856558 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ad24a-527b-45f0-a273-768643660bba None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:57.859507 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efb720e-3990-4dea-b57d-e97e032cb4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:57.860178 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb720e-3990-4dea-b57d-e97e032cb4a4 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:57.860484 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb720e-3990-4dea-b57d-e97e032cb4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:57.860805 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb720e-3990-4dea-b57d-e97e032cb4a4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:57.873464 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efb720e-3990-4dea-b57d-e97e032cb4a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:57.878811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-2efb720e-3990-4dea-b57d-e97e032cb4a4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 14:06:57.878811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:57.878811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:57.878811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:57.878811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:57.878811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:57.878811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:57.878811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:57.878811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:57.878811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:57.878811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:57.878811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:57.878811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:57.878811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:57.878811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:57.878811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:57.879697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:57.879697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:57.879697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:57.879697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:57.879697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:57.879697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:57.879697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:57.879697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:57.879697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:57.879697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:57.879697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:57.879697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:57.879697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:57.879697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:57.879697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:57.879697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:57.880853 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:57.880853 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:57.880853 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:57.880853 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:57.880853 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:57.880853 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:57.880853 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:57.880853 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:57.880853 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:57.880853 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:57.880853 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 14:06:57.880853 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:06:57.880853 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 174/363] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Aug 5 14:06:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:06:57.886573 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efb720e-3990-4dea-b57d-e97e032cb4a4 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:57.988921 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:58.017833 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efb720e-3990-4dea-b57d-e97e032cb4a4 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:58.030078 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 190/364] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:06:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 14:06:58.038352 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ad24a-527b-45f0-a273-768643660bba None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:58.066670 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d08dd2-876d-4e52-af07-33a99ae5bf87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:58.067565 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d08dd2-876d-4e52-af07-33a99ae5bf87 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:58.067956 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d08dd2-876d-4e52-af07-33a99ae5bf87 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:58.068245 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d08dd2-876d-4e52-af07-33a99ae5bf87 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:58.069635 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d08dd2-876d-4e52-af07-33a99ae5bf87 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:58.069970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d08dd2-876d-4e52-af07-33a99ae5bf87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:58.071376 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d08dd2-876d-4e52-af07-33a99ae5bf87 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:58.081587 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 175/365] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:06:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:58.086435 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:58.088451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d08dd2-876d-4e52-af07-33a99ae5bf87 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:58.198065 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ad24a-527b-45f0-a273-768643660bba None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:58.206419 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d08dd2-876d-4e52-af07-33a99ae5bf87 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:58.236298 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd553a61-af1d-4d7a-a3b0-10d4e8397c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:58.237029 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd553a61-af1d-4d7a-a3b0-10d4e8397c42 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:58.237248 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd553a61-af1d-4d7a-a3b0-10d4e8397c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:58.237518 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd553a61-af1d-4d7a-a3b0-10d4e8397c42 None admin] PATH_INFO: `/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:58.238693 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd553a61-af1d-4d7a-a3b0-10d4e8397c42 None admin] RBAC: Authorizing `identity:get_project(project_id=ecbbd8ae8eed4ce5b441221e8e17663b)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:58.240305 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd553a61-af1d-4d7a-a3b0-10d4e8397c42 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:58.242356 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 191/366] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:06:58 2022] GET /identity/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:58.243612 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c1b06-6d67-4536-a70d-52aeae9a051d None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:58.243889 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c1b06-6d67-4536-a70d-52aeae9a051d None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:58.244160 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c1b06-6d67-4536-a70d-52aeae9a051d None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:58.248500 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 192/367] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:06:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:06:58.249288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c1b06-6d67-4536-a70d-52aeae9a051d None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:58.253175 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c484b4cd-448b-4d03-83e7-04a8b323d6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:58.253804 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c484b4cd-448b-4d03-83e7-04a8b323d6ee None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:58.254081 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c484b4cd-448b-4d03-83e7-04a8b323d6ee None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:58.254355 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c484b4cd-448b-4d03-83e7-04a8b323d6ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:58.268173 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:58.350641 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c484b4cd-448b-4d03-83e7-04a8b323d6ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:58.352167 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c484b4cd-448b-4d03-83e7-04a8b323d6ee None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:58.395349 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:58.410278 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c1b06-6d67-4536-a70d-52aeae9a051d None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:58.456717 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7e37a-0a68-4637-a9b1-7c6f5d75ab00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:58.458166 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7e37a-0a68-4637-a9b1-7c6f5d75ab00 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:58.459462 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7e37a-0a68-4637-a9b1-7c6f5d75ab00 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:58.459974 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7e37a-0a68-4637-a9b1-7c6f5d75ab00 None admin] PATH_INFO: `/v3/groups/674c9fb834714b89a516d5f2cc7d5005` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:58.462056 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7e37a-0a68-4637-a9b1-7c6f5d75ab00 None admin] RBAC: Authorizing `identity:get_group(group_id=674c9fb834714b89a516d5f2cc7d5005)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:58.464209 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7e37a-0a68-4637-a9b1-7c6f5d75ab00 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:58.467256 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 193/368] 10.222.0.26 () {56 vars in 1180 bytes} [Fri Aug 5 14:06:58 2022] GET /identity/v3/groups/674c9fb834714b89a516d5f2cc7d5005 => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:58.468680 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149cb351-6da5-44ea-97f0-7b33eaf8ba07 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:58.469210 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149cb351-6da5-44ea-97f0-7b33eaf8ba07 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:58.469756 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149cb351-6da5-44ea-97f0-7b33eaf8ba07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:58.477185 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 176/369] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:06:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 14:06:58.478040 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c484b4cd-448b-4d03-83e7-04a8b323d6ee None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:58.495782 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-149cb351-6da5-44ea-97f0-7b33eaf8ba07 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:06:58.497041 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-149cb351-6da5-44ea-97f0-7b33eaf8ba07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:06:58.537626 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:58.581486 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 194/370] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:06:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:06:58.583035 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149cb351-6da5-44ea-97f0-7b33eaf8ba07 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:58.621497 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c484b4cd-448b-4d03-83e7-04a8b323d6ee None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:58.665697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4ca3b-d7cc-46fc-a018-d1732fd8c4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:58.666443 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4ca3b-d7cc-46fc-a018-d1732fd8c4e3 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:58.666784 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4ca3b-d7cc-46fc-a018-d1732fd8c4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:58.667141 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4ca3b-d7cc-46fc-a018-d1732fd8c4e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:58.668002 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd4ca3b-d7cc-46fc-a018-d1732fd8c4e3 None admin] RBAC: Adding query filter params (group.id=674c9fb834714b89a516d5f2cc7d5005, role.id=29967df108484a30a4c006cde2ae2940, scope.project.id=ecbbd8ae8eed4ce5b441221e8e17663b) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:58.668377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd4ca3b-d7cc-46fc-a018-d1732fd8c4e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:58.669945 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd4ca3b-d7cc-46fc-a018-d1732fd8c4e3 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:58.676141 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 177/371] 10.222.0.26 () {56 vars in 1376 bytes} [Fri Aug 5 14:06:58 2022] GET /identity/v3/role_assignments?group.id=674c9fb834714b89a516d5f2cc7d5005&scope.project.id=ecbbd8ae8eed4ce5b441221e8e17663b&role.id=29967df108484a30a4c006cde2ae2940 => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:58.677875 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4ca3b-d7cc-46fc-a018-d1732fd8c4e3 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:58.697052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:58.710627 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149cb351-6da5-44ea-97f0-7b33eaf8ba07 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:58.769830 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6326e9-757a-4dce-bb28-7c68f3704609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:58.770491 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6326e9-757a-4dce-bb28-7c68f3704609 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:58.770792 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6326e9-757a-4dce-bb28-7c68f3704609 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:58.771071 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6326e9-757a-4dce-bb28-7c68f3704609 None admin] PATH_INFO: `/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:58.772351 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6326e9-757a-4dce-bb28-7c68f3704609 None admin] RBAC: Authorizing `identity:get_project(project_id=ecbbd8ae8eed4ce5b441221e8e17663b)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:58.773866 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6326e9-757a-4dce-bb28-7c68f3704609 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:58.775704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 195/372] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:06:58 2022] GET /identity/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:58.781372 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6326e9-757a-4dce-bb28-7c68f3704609 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:58.820553 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4ca3b-d7cc-46fc-a018-d1732fd8c4e3 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:58.822895 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:58.864075 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b69263e-4fd2-4427-8898-f6568959266b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:58.866101 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b69263e-4fd2-4427-8898-f6568959266b None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:58.866612 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b69263e-4fd2-4427-8898-f6568959266b None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:58.867098 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b69263e-4fd2-4427-8898-f6568959266b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:58.881464 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b69263e-4fd2-4427-8898-f6568959266b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:06:58.886919 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-2b69263e-4fd2-4427-8898-f6568959266b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:06:58.886919 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:06:58.886919 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:06:58.886919 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:06:58.886919 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:06:58.886919 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:06:58.886919 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:06:58.886919 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:06:58.886919 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:06:58.886919 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:06:58.886919 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:06:58.886919 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:06:58.886919 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:06:58.886919 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:06:58.886919 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:06:58.886919 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:06:58.887825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:06:58.887825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:06:58.887825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:06:58.887825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:06:58.887825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:06:58.887825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:06:58.887825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:06:58.887825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:06:58.887825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:06:58.887825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:06:58.887825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:06:58.887825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:06:58.887825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:06:58.887825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:06:58.887825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:06:58.887825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:06:58.888647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:06:58.888647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:06:58.888647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:06:58.888647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:06:58.888647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:06:58.888647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:06:58.888647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:06:58.888647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:06:58.888647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:06:58.888647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:06:58.888647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:06:58.888647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:06:58.891341 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 178/373] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Aug 5 14:06:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:06:58.899069 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b69263e-4fd2-4427-8898-f6568959266b None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:58.910539 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6326e9-757a-4dce-bb28-7c68f3704609 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:58.964752 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20216a-a8ff-4d11-8010-d2a426660d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:58.965433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20216a-a8ff-4d11-8010-d2a426660d92 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:58.965699 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20216a-a8ff-4d11-8010-d2a426660d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:58.965973 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20216a-a8ff-4d11-8010-d2a426660d92 None admin] PATH_INFO: `/v3/groups/674c9fb834714b89a516d5f2cc7d5005` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:58.967188 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d20216a-a8ff-4d11-8010-d2a426660d92 None admin] RBAC: Authorizing `identity:get_group(group_id=674c9fb834714b89a516d5f2cc7d5005)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:58.968619 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d20216a-a8ff-4d11-8010-d2a426660d92 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:58.970575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 196/374] 10.222.0.26 () {56 vars in 1180 bytes} [Fri Aug 5 14:06:58 2022] GET /identity/v3/groups/674c9fb834714b89a516d5f2cc7d5005 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:58.976851 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20216a-a8ff-4d11-8010-d2a426660d92 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:59.007600 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:59.033955 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b69263e-4fd2-4427-8898-f6568959266b None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:59.072737 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6fc0ed-8704-41f4-ae14-4c1c169a4ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:59.073699 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6fc0ed-8704-41f4-ae14-4c1c169a4ead None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:59.074179 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6fc0ed-8704-41f4-ae14-4c1c169a4ead None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:59.074678 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6fc0ed-8704-41f4-ae14-4c1c169a4ead None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:59.075592 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6fc0ed-8704-41f4-ae14-4c1c169a4ead None admin] RBAC: Adding query filter params (name=admin) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:59.076096 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6fc0ed-8704-41f4-ae14-4c1c169a4ead None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:59.077712 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6fc0ed-8704-41f4-ae14-4c1c169a4ead None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:59.086720 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20216a-a8ff-4d11-8010-d2a426660d92 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:59.087609 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 179/375] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:06:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:59.097924 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6fc0ed-8704-41f4-ae14-4c1c169a4ead None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:59.133892 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:59.148390 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e48a2f-861f-48ca-825f-e7e92f433bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:59.148390 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e48a2f-861f-48ca-825f-e7e92f433bee None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:59.148390 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e48a2f-861f-48ca-825f-e7e92f433bee None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:59.148390 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e48a2f-861f-48ca-825f-e7e92f433bee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:59.148818 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e48a2f-861f-48ca-825f-e7e92f433bee None admin] RBAC: Adding query filter params (group.id=674c9fb834714b89a516d5f2cc7d5005, role.id=199fd3093a824ba585b1135c8bf363be, scope.project.id=ecbbd8ae8eed4ce5b441221e8e17663b) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:06:59.149083 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e48a2f-861f-48ca-825f-e7e92f433bee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:59.150570 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e48a2f-861f-48ca-825f-e7e92f433bee None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:59.160098 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 197/376] 10.222.0.26 () {56 vars in 1376 bytes} [Fri Aug 5 14:06:58 2022] GET /identity/v3/role_assignments?group.id=674c9fb834714b89a516d5f2cc7d5005&scope.project.id=ecbbd8ae8eed4ce5b441221e8e17663b&role.id=199fd3093a824ba585b1135c8bf363be => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:06:59.255789 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6fc0ed-8704-41f4-ae14-4c1c169a4ead None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:59.301242 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad1d7d-6a40-42b4-989a-b966de537b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:59.301778 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad1d7d-6a40-42b4-989a-b966de537b52 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:59.302599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad1d7d-6a40-42b4-989a-b966de537b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:59.302868 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad1d7d-6a40-42b4-989a-b966de537b52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:59.387492 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ad1d7d-6a40-42b4-989a-b966de537b52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:59.389026 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ad1d7d-6a40-42b4-989a-b966de537b52 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:59.419280 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:59.524149 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 180/377] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:06:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 14:06:59.532636 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e48a2f-861f-48ca-825f-e7e92f433bee None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:59.639270 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:59.654642 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e48a2f-861f-48ca-825f-e7e92f433bee None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:59.702392 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941e604-2cd4-404f-8644-b1defcde0d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:59.703040 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941e604-2cd4-404f-8644-b1defcde0d45 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:59.703316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941e604-2cd4-404f-8644-b1defcde0d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:59.703580 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941e604-2cd4-404f-8644-b1defcde0d45 None admin] PATH_INFO: `/v3/projects/b40e0244e98a49699194b094ad410d1e` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:59.704788 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6941e604-2cd4-404f-8644-b1defcde0d45 None admin] RBAC: Authorizing `identity:get_project(project_id=b40e0244e98a49699194b094ad410d1e)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:59.706235 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6941e604-2cd4-404f-8644-b1defcde0d45 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:59.708856 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 198/378] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:06:59 2022] GET /identity/v3/projects/b40e0244e98a49699194b094ad410d1e => generated 344 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:59.714441 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad1d7d-6a40-42b4-989a-b966de537b52 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:06:59.846638 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad1d7d-6a40-42b4-989a-b966de537b52 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:06:59.871748 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:06:59.885884 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b3ebb-aca1-434c-94c8-006bc8e188a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:06:59.886539 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b3ebb-aca1-434c-94c8-006bc8e188a7 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:06:59.886818 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b3ebb-aca1-434c-94c8-006bc8e188a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:06:59.887115 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b3ebb-aca1-434c-94c8-006bc8e188a7 None admin] PATH_INFO: `/v3/groups/c78073879dc54a9497fd340df09dbb10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:06:59.894640 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1b3ebb-aca1-434c-94c8-006bc8e188a7 None admin] RBAC: Authorizing `identity:get_group(group_id=c78073879dc54a9497fd340df09dbb10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:06:59.897304 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1b3ebb-aca1-434c-94c8-006bc8e188a7 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:06:59.899321 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 181/379] 10.222.0.26 () {56 vars in 1180 bytes} [Fri Aug 5 14:06:59 2022] GET /identity/v3/groups/c78073879dc54a9497fd340df09dbb10 => generated 229 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:06:59.906122 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941e604-2cd4-404f-8644-b1defcde0d45 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:00.014972 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4a23e-e143-4f11-ae9f-772d8b5c71e7 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:00.015288 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4a23e-e143-4f11-ae9f-772d8b5c71e7 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:00.015525 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4a23e-e143-4f11-ae9f-772d8b5c71e7 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:00.017038 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 182/380] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:00.024332 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb103da-4364-4a3e-b492-805c782e1fa3 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:00.024661 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb103da-4364-4a3e-b492-805c782e1fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:00.024937 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb103da-4364-4a3e-b492-805c782e1fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:00.031695 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941e604-2cd4-404f-8644-b1defcde0d45 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:00.065472 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-ceb103da-4364-4a3e-b492-805c782e1fa3 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:00.066630 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ceb103da-4364-4a3e-b492-805c782e1fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:00.078288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4b062-9be7-4d2a-a85e-752067201813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:00.079245 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4b062-9be7-4d2a-a85e-752067201813 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:00.079702 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4b062-9be7-4d2a-a85e-752067201813 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:00.080151 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4b062-9be7-4d2a-a85e-752067201813 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:00.081245 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4b062-9be7-4d2a-a85e-752067201813 None admin] RBAC: Adding query filter params (group.id=c78073879dc54a9497fd340df09dbb10, role.id=299c794c7acb4a5ca36288c3191aad10, scope.project.id=b40e0244e98a49699194b094ad410d1e) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:00.081812 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4b062-9be7-4d2a-a85e-752067201813 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:00.084549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4b062-9be7-4d2a-a85e-752067201813 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:00.094694 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 199/381] 10.222.0.26 () {56 vars in 1376 bytes} [Fri Aug 5 14:06:59 2022] GET /identity/v3/role_assignments?group.id=c78073879dc54a9497fd340df09dbb10&scope.project.id=b40e0244e98a49699194b094ad410d1e&role.id=299c794c7acb4a5ca36288c3191aad10 => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:00.174767 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 183/382] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:07:00.205801 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4b062-9be7-4d2a-a85e-752067201813 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:00.236921 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:00.323436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4b062-9be7-4d2a-a85e-752067201813 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:00.361035 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80987647-b7fa-4917-93ef-7d35057651cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:00.361692 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80987647-b7fa-4917-93ef-7d35057651cf None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:00.362024 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80987647-b7fa-4917-93ef-7d35057651cf None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:00.362303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80987647-b7fa-4917-93ef-7d35057651cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:00.374943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80987647-b7fa-4917-93ef-7d35057651cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:07:00.381485 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-80987647-b7fa-4917-93ef-7d35057651cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 14:07:00.381485 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:00.381485 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:00.381485 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:00.381485 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:00.381485 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:00.381485 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:00.381485 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:00.381485 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:00.381485 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:00.381485 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:00.381485 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:00.381485 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:07:00.381485 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:07:00.381485 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:07:00.381485 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:07:00.382480 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:07:00.382480 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:07:00.382480 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:07:00.382480 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:07:00.382480 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:00.382480 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:00.382480 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:00.382480 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:00.382480 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:00.382480 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:00.382480 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:00.382480 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:00.382480 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:00.382480 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:00.382480 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:00.382480 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:00.383280 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:00.383280 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:00.383280 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:00.383280 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:00.383280 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:07:00.383280 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:07:00.383280 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:07:00.383280 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:07:00.383280 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:07:00.383280 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:07:00.383280 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 14:07:00.383280 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:00.383280 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 200/383] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:07:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:00.390468 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb103da-4364-4a3e-b492-805c782e1fa3 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:00.480094 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cceee6-84de-4dd3-ab44-bcd8613705cd None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:00.480094 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cceee6-84de-4dd3-ab44-bcd8613705cd None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:00.480094 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cceee6-84de-4dd3-ab44-bcd8613705cd None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:00.480809 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 201/384] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:07:00.485989 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7e9ff-771c-4ff3-b6e8-5110a5931deb None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:00.486517 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7e9ff-771c-4ff3-b6e8-5110a5931deb None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:00.486995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7e9ff-771c-4ff3-b6e8-5110a5931deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:00.507442 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-72f7e9ff-771c-4ff3-b6e8-5110a5931deb None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:00.509019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-72f7e9ff-771c-4ff3-b6e8-5110a5931deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:00.513503 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb103da-4364-4a3e-b492-805c782e1fa3 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:00.540045 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:00.573469 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372115e-3eeb-400a-b4a7-38967f35b056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:00.573857 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372115e-3eeb-400a-b4a7-38967f35b056 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:00.574103 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372115e-3eeb-400a-b4a7-38967f35b056 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:00.574378 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372115e-3eeb-400a-b4a7-38967f35b056 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:00.575088 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b372115e-3eeb-400a-b4a7-38967f35b056 None admin] RBAC: Adding query filter params (name=member) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:00.575390 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b372115e-3eeb-400a-b4a7-38967f35b056 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:00.576791 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b372115e-3eeb-400a-b4a7-38967f35b056 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:00.592036 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 184/385] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Aug 5 14:07:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:00.603043 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 202/386] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 14:07:00.622078 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372115e-3eeb-400a-b4a7-38967f35b056 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:00.626263 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7e9ff-771c-4ff3-b6e8-5110a5931deb None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:00.646490 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:00.743588 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7e9ff-771c-4ff3-b6e8-5110a5931deb None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:00.782070 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482a751b-2ecc-4b3b-a4b2-a92bc71f5b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:00.783565 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a751b-2ecc-4b3b-a4b2-a92bc71f5b99 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:00.784049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a751b-2ecc-4b3b-a4b2-a92bc71f5b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:00.784599 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a751b-2ecc-4b3b-a4b2-a92bc71f5b99 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:00.796276 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482a751b-2ecc-4b3b-a4b2-a92bc71f5b99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:07:00.801869 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-482a751b-2ecc-4b3b-a4b2-a92bc71f5b99 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 14:07:00.801869 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:00.801869 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:00.801869 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:00.801869 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:00.801869 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:00.801869 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:00.801869 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:00.801869 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:00.801869 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:00.801869 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:00.801869 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:00.801869 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:07:00.801869 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:07:00.801869 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:07:00.801869 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:07:00.803560 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:07:00.803560 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:07:00.803560 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:07:00.803560 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:07:00.803560 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:00.803560 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:00.803560 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:00.803560 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:00.803560 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:00.803560 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:00.803560 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:00.803560 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:00.803560 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:00.803560 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:00.803560 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:00.803560 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:00.804379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:00.804379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:00.804379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:00.804379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:00.804379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:07:00.804379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:07:00.804379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:07:00.804379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:07:00.804379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:07:00.804379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:07:00.804379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 14:07:00.804379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:00.807164 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 203/387] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Aug 5 14:07:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:00.813345 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482a751b-2ecc-4b3b-a4b2-a92bc71f5b99 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:00.823968 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372115e-3eeb-400a-b4a7-38967f35b056 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:00.896606 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed771d4e-2e7f-4af9-8980-e2f09a6aa3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:00.897659 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed771d4e-2e7f-4af9-8980-e2f09a6aa3c8 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:00.898215 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed771d4e-2e7f-4af9-8980-e2f09a6aa3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:00.898686 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed771d4e-2e7f-4af9-8980-e2f09a6aa3c8 None admin] PATH_INFO: `/v3/groups/674c9fb834714b89a516d5f2cc7d5005` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:00.901303 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed771d4e-2e7f-4af9-8980-e2f09a6aa3c8 None admin] RBAC: Authorizing `identity:get_group(group_id=674c9fb834714b89a516d5f2cc7d5005)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:00.903298 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed771d4e-2e7f-4af9-8980-e2f09a6aa3c8 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:00.907569 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 185/388] 10.222.0.26 () {56 vars in 1180 bytes} [Fri Aug 5 14:07:00 2022] GET /identity/v3/groups/674c9fb834714b89a516d5f2cc7d5005 => generated 226 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:00.915221 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed771d4e-2e7f-4af9-8980-e2f09a6aa3c8 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:00.955620 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:01.002614 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:01.012680 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482a751b-2ecc-4b3b-a4b2-a92bc71f5b99 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:01.079766 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7092f40-949c-400c-939b-951db4771c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:01.080394 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7092f40-949c-400c-939b-951db4771c97 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:01.081744 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7092f40-949c-400c-939b-951db4771c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:01.081744 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7092f40-949c-400c-939b-951db4771c97 None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:01.081744 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7092f40-949c-400c-939b-951db4771c97 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:01.081937 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7092f40-949c-400c-939b-951db4771c97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:01.083345 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7092f40-949c-400c-939b-951db4771c97 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:01.087577 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed771d4e-2e7f-4af9-8980-e2f09a6aa3c8 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:01.099318 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 204/389] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:07:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:01.100277 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7092f40-949c-400c-939b-951db4771c97 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:01.135205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a7087-bc18-42ee-a47a-578fca870102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:01.136294 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a7087-bc18-42ee-a47a-578fca870102 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:01.136879 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a7087-bc18-42ee-a47a-578fca870102 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:01.137357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a7087-bc18-42ee-a47a-578fca870102 None admin] PATH_INFO: `/v3/projects/580a155c67b742b9bc398f50556f26cf` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:01.138934 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899a7087-bc18-42ee-a47a-578fca870102 None admin] RBAC: Authorizing `identity:get_project(project_id=580a155c67b742b9bc398f50556f26cf)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:01.140682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899a7087-bc18-42ee-a47a-578fca870102 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:01.142918 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 186/390] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:07:00 2022] GET /identity/v3/projects/580a155c67b742b9bc398f50556f26cf => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:01.151371 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a7087-bc18-42ee-a47a-578fca870102 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:01.291546 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a7087-bc18-42ee-a47a-578fca870102 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:01.321389 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7092f40-949c-400c-939b-951db4771c97 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:01.326022 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:01.365217 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dbe7a1-dff2-4721-a590-75a867ad8247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:01.367500 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbe7a1-dff2-4721-a590-75a867ad8247 None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:01.368158 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbe7a1-dff2-4721-a590-75a867ad8247 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:01.369718 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbe7a1-dff2-4721-a590-75a867ad8247 None admin] PATH_INFO: `/v3/projects/580a155c67b742b9bc398f50556f26cf/groups/674c9fb834714b89a516d5f2cc7d5005/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:01.373076 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dbe7a1-dff2-4721-a590-75a867ad8247 None admin] RBAC: Authorizing `identity:create_grant(project_id=580a155c67b742b9bc398f50556f26cf, group_id=674c9fb834714b89a516d5f2cc7d5005, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:01.374515 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dbe7a1-dff2-4721-a590-75a867ad8247 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:01.388494 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d2c97a-7820-46ab-b42c-9c239477088a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:01.389466 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2c97a-7820-46ab-b42c-9c239477088a None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:01.389908 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2c97a-7820-46ab-b42c-9c239477088a None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:01.390354 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2c97a-7820-46ab-b42c-9c239477088a None admin] PATH_INFO: `/v3/groups/674c9fb834714b89a516d5f2cc7d5005` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:01.391507 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d2c97a-7820-46ab-b42c-9c239477088a None admin] RBAC: Authorizing `identity:get_group(group_id=674c9fb834714b89a516d5f2cc7d5005)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:01.396509 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d2c97a-7820-46ab-b42c-9c239477088a None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:01.396509 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 205/391] 10.222.0.26 () {56 vars in 1180 bytes} [Fri Aug 5 14:07:01 2022] GET /identity/v3/groups/674c9fb834714b89a516d5f2cc7d5005 => generated 226 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:01.396509 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970b8b0-3a80-4e2d-87e0-0a280ce22eb3 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:01.396921 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970b8b0-3a80-4e2d-87e0-0a280ce22eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:01.397270 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970b8b0-3a80-4e2d-87e0-0a280ce22eb3 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:01.398550 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 206/392] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:01.402035 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a970b8b0-3a80-4e2d-87e0-0a280ce22eb3 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:01.406151 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:01.453662 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 187/393] 10.222.0.26 () {58 vars in 1442 bytes} [Fri Aug 5 14:07:01 2022] PUT /identity/v3/projects/580a155c67b742b9bc398f50556f26cf/groups/674c9fb834714b89a516d5f2cc7d5005/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:07:01.456221 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4055a-0102-45cf-8e92-3f30568dc012 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:01.457064 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4055a-0102-45cf-8e92-3f30568dc012 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:01.457622 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4055a-0102-45cf-8e92-3f30568dc012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:01.487293 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-9ed4055a-0102-45cf-8e92-3f30568dc012 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:01.488907 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-9ed4055a-0102-45cf-8e92-3f30568dc012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:01.574011 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a970b8b0-3a80-4e2d-87e0-0a280ce22eb3 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:01.602038 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 188/394] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:07:01.646904 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8739869-2965-4d3a-b600-1ab5b29c329c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:01.647555 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8739869-2965-4d3a-b600-1ab5b29c329c None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:01.647838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8739869-2965-4d3a-b600-1ab5b29c329c None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:01.648114 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8739869-2965-4d3a-b600-1ab5b29c329c None admin] PATH_INFO: `/v3/projects/580a155c67b742b9bc398f50556f26cf` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:01.650177 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8739869-2965-4d3a-b600-1ab5b29c329c None admin] RBAC: Authorizing `identity:get_project(project_id=580a155c67b742b9bc398f50556f26cf)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:01.651702 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8739869-2965-4d3a-b600-1ab5b29c329c None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:01.653857 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 207/395] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:07:01 2022] GET /identity/v3/projects/580a155c67b742b9bc398f50556f26cf => generated 298 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:01.657613 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed4055a-0102-45cf-8e92-3f30568dc012 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:01.659757 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8739869-2965-4d3a-b600-1ab5b29c329c None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:01.770038 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:01.815539 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed4055a-0102-45cf-8e92-3f30568dc012 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:01.816250 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8739869-2965-4d3a-b600-1ab5b29c329c None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:01.825028 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:01.862065 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b58ee-3aa7-461a-a3b7-7a91507015f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:01.863724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b58ee-3aa7-461a-a3b7-7a91507015f1 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:01.863724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b58ee-3aa7-461a-a3b7-7a91507015f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:01.863724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b58ee-3aa7-461a-a3b7-7a91507015f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:01.865369 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893220a-1c0d-4b78-827c-e7db52553cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:01.866036 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893220a-1c0d-4b78-827c-e7db52553cfb None admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:01.866313 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893220a-1c0d-4b78-827c-e7db52553cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:01.866585 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893220a-1c0d-4b78-827c-e7db52553cfb None admin] PATH_INFO: `/v3/projects/580a155c67b742b9bc398f50556f26cf/groups/674c9fb834714b89a516d5f2cc7d5005/roles/199fd3093a824ba585b1135c8bf363be` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:01.869475 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7893220a-1c0d-4b78-827c-e7db52553cfb None admin] RBAC: Authorizing `identity:create_grant(project_id=580a155c67b742b9bc398f50556f26cf, group_id=674c9fb834714b89a516d5f2cc7d5005, role_id=199fd3093a824ba585b1135c8bf363be)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:01.871142 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7893220a-1c0d-4b78-827c-e7db52553cfb None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:01.877728 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8b58ee-3aa7-461a-a3b7-7a91507015f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:07:01.884379 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-0c8b58ee-3aa7-461a-a3b7-7a91507015f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 14:07:01.884379 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:01.884379 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:01.884379 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:01.884379 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:01.884379 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:01.884379 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:01.884379 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:01.884379 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:01.884379 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:01.884379 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:01.884379 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:01.884379 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:07:01.884379 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:07:01.884379 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:07:01.884379 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:07:01.888856 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:07:01.888856 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:07:01.888856 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:07:01.888856 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:07:01.888856 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:01.888856 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:01.888856 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:01.888856 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:01.888856 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:01.888856 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:01.888856 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:01.888856 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:01.888856 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:01.888856 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:01.888856 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:01.888856 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:01.889666 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:01.889666 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:01.889666 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:01.889666 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:01.889666 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:07:01.889666 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:07:01.889666 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:07:01.889666 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:07:01.889666 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:07:01.889666 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:07:01.889666 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 14:07:01.889666 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:01.889666 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 189/396] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:07:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:01.891994 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b58ee-3aa7-461a-a3b7-7a91507015f1 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:01.975728 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 208/397] 10.222.0.26 () {58 vars in 1442 bytes} [Fri Aug 5 14:07:01 2022] PUT /identity/v3/projects/580a155c67b742b9bc398f50556f26cf/groups/674c9fb834714b89a516d5f2cc7d5005/roles/199fd3093a824ba585b1135c8bf363be => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:07:02.022004 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b58ee-3aa7-461a-a3b7-7a91507015f1 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:02.068082 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab4499f-d7c9-42a4-81cb-3dccaf97bd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:02.069420 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4499f-d7c9-42a4-81cb-3dccaf97bd70 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:02.069899 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4499f-d7c9-42a4-81cb-3dccaf97bd70 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:02.070410 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4499f-d7c9-42a4-81cb-3dccaf97bd70 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:02.071443 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab4499f-d7c9-42a4-81cb-3dccaf97bd70 None admin] RBAC: Adding query filter params (name=member) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:02.071952 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab4499f-d7c9-42a4-81cb-3dccaf97bd70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:02.073608 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab4499f-d7c9-42a4-81cb-3dccaf97bd70 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:02.083931 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 190/398] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Aug 5 14:07:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:02.088927 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893220a-1c0d-4b78-827c-e7db52553cfb None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:02.215286 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893220a-1c0d-4b78-827c-e7db52553cfb None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:02.250750 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:02.256837 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31730272-424c-434e-ab89-d51ed3be4801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:02.257504 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31730272-424c-434e-ab89-d51ed3be4801 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:02.257785 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31730272-424c-434e-ab89-d51ed3be4801 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:02.258062 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31730272-424c-434e-ab89-d51ed3be4801 None admin] PATH_INFO: `/v3/groups/674c9fb834714b89a516d5f2cc7d5005` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:02.259308 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31730272-424c-434e-ab89-d51ed3be4801 None admin] RBAC: Authorizing `identity:get_group(group_id=674c9fb834714b89a516d5f2cc7d5005)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:02.260800 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31730272-424c-434e-ab89-d51ed3be4801 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:02.262803 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 209/399] 10.222.0.26 () {56 vars in 1180 bytes} [Fri Aug 5 14:07:02 2022] GET /identity/v3/groups/674c9fb834714b89a516d5f2cc7d5005 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:02.268685 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab4499f-d7c9-42a4-81cb-3dccaf97bd70 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:02.301432 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:02.349539 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0365f3c0-04f3-495f-9c2c-14baf15b7cb5 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:02.349824 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0365f3c0-04f3-495f-9c2c-14baf15b7cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:02.350160 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0365f3c0-04f3-495f-9c2c-14baf15b7cb5 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:02.351545 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 210/400] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:07:02.365077 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1068c6f-b24e-4bdc-9d72-b8a3d78bede4 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:02.365474 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1068c6f-b24e-4bdc-9d72-b8a3d78bede4 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:02.365644 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1068c6f-b24e-4bdc-9d72-b8a3d78bede4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:02.388915 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab4499f-d7c9-42a4-81cb-3dccaf97bd70 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:02.390270 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-d1068c6f-b24e-4bdc-9d72-b8a3d78bede4 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:02.391478 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-d1068c6f-b24e-4bdc-9d72-b8a3d78bede4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:02.433984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca387035-cd23-4d68-b034-53d6c18a4272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:02.434621 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca387035-cd23-4d68-b034-53d6c18a4272 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:02.434900 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca387035-cd23-4d68-b034-53d6c18a4272 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:02.435170 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca387035-cd23-4d68-b034-53d6c18a4272 None admin] PATH_INFO: `/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:02.438119 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca387035-cd23-4d68-b034-53d6c18a4272 None admin] RBAC: Authorizing `identity:get_project(project_id=ecbbd8ae8eed4ce5b441221e8e17663b)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:02.439942 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca387035-cd23-4d68-b034-53d6c18a4272 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:02.441611 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 191/401] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:07:02 2022] GET /identity/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:02.449332 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca387035-cd23-4d68-b034-53d6c18a4272 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:02.483668 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 211/402] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:07:02.506096 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1068c6f-b24e-4bdc-9d72-b8a3d78bede4 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:02.617957 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1068c6f-b24e-4bdc-9d72-b8a3d78bede4 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:02.623888 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:02.629600 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca387035-cd23-4d68-b034-53d6c18a4272 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:02.650896 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:02.659474 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736b81e-e213-4d6c-8cf6-c30c937ecf09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:02.660390 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736b81e-e213-4d6c-8cf6-c30c937ecf09 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:02.660859 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736b81e-e213-4d6c-8cf6-c30c937ecf09 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:02.661312 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736b81e-e213-4d6c-8cf6-c30c937ecf09 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:02.676489 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736b81e-e213-4d6c-8cf6-c30c937ecf09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:07:02.677560 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f2ef6-f88f-4935-8517-dc69a765b1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:02.678085 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f2ef6-f88f-4935-8517-dc69a765b1c7 None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:02.678371 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f2ef6-f88f-4935-8517-dc69a765b1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:02.678650 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f2ef6-f88f-4935-8517-dc69a765b1c7 None admin] PATH_INFO: `/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b/groups/674c9fb834714b89a516d5f2cc7d5005/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:02.681687 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f2ef6-f88f-4935-8517-dc69a765b1c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=ecbbd8ae8eed4ce5b441221e8e17663b, group_id=674c9fb834714b89a516d5f2cc7d5005, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:02.683232 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f2ef6-f88f-4935-8517-dc69a765b1c7 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:02.683477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-4736b81e-e213-4d6c-8cf6-c30c937ecf09 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 14:07:02.683477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:02.683477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:02.683477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:02.683477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:02.683477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:02.683477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:02.683477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:02.683477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:02.683477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:02.683477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:02.683477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:02.683477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:07:02.683477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:07:02.683477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:07:02.683477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:07:02.684587 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:07:02.684587 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:07:02.684587 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:07:02.684587 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:07:02.684587 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:02.684587 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:02.684587 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:02.684587 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:02.684587 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:02.684587 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:02.684587 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:02.684587 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:02.684587 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:02.684587 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:02.684587 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:02.684587 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:02.685583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:02.685583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:02.685583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:02.685583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:02.685583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:07:02.685583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:07:02.685583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:07:02.685583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:07:02.685583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:07:02.685583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:07:02.685583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 14:07:02.685583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:02.692231 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 212/403] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Aug 5 14:07:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:07:02.695154 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078eb5c-a576-40c8-9676-7b288979d156 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:02.695611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078eb5c-a576-40c8-9676-7b288979d156 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:02.696045 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078eb5c-a576-40c8-9676-7b288979d156 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:02.697944 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 213/404] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:02.698807 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d078eb5c-a576-40c8-9676-7b288979d156 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:02.805156 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d078eb5c-a576-40c8-9676-7b288979d156 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:02.845430 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be551645-56e8-45b3-abf9-c4a65f968dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:02.846057 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be551645-56e8-45b3-abf9-c4a65f968dfa None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:02.846334 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be551645-56e8-45b3-abf9-c4a65f968dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:02.846608 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be551645-56e8-45b3-abf9-c4a65f968dfa None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:02.847376 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be551645-56e8-45b3-abf9-c4a65f968dfa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:02.847681 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be551645-56e8-45b3-abf9-c4a65f968dfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:02.849195 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be551645-56e8-45b3-abf9-c4a65f968dfa None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:02.857770 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 214/405] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:07:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:02.863455 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4879ac-0ec2-4443-a9aa-d412816136dc None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:02.863455 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4879ac-0ec2-4443-a9aa-d412816136dc None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:02.863455 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4879ac-0ec2-4443-a9aa-d412816136dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:02.878486 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-7f4879ac-0ec2-4443-a9aa-d412816136dc None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:02.879747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-7f4879ac-0ec2-4443-a9aa-d412816136dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:02.954871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 192/406] 10.222.0.26 () {58 vars in 1442 bytes} [Fri Aug 5 14:07:02 2022] PUT /identity/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b/groups/674c9fb834714b89a516d5f2cc7d5005/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:07:02.955711 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f2ef6-f88f-4935-8517-dc69a765b1c7 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:02.963217 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 215/407] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:07:02.980772 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:02.987686 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4879ac-0ec2-4443-a9aa-d412816136dc None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:03.140143 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f2ef6-f88f-4935-8517-dc69a765b1c7 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:03.181655 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4879ac-0ec2-4443-a9aa-d412816136dc None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:03.181871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407557e6-273e-4a8c-9d0b-365a721aacd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:03.182418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407557e6-273e-4a8c-9d0b-365a721aacd8 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:03.182699 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407557e6-273e-4a8c-9d0b-365a721aacd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:03.182973 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407557e6-273e-4a8c-9d0b-365a721aacd8 None admin] PATH_INFO: `/v3/groups/674c9fb834714b89a516d5f2cc7d5005` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:03.184148 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407557e6-273e-4a8c-9d0b-365a721aacd8 None admin] RBAC: Authorizing `identity:get_group(group_id=674c9fb834714b89a516d5f2cc7d5005)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:03.186137 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407557e6-273e-4a8c-9d0b-365a721aacd8 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:03.187198 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 193/408] 10.222.0.26 () {56 vars in 1180 bytes} [Fri Aug 5 14:07:02 2022] GET /identity/v3/groups/674c9fb834714b89a516d5f2cc7d5005 => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:03.193065 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407557e6-273e-4a8c-9d0b-365a721aacd8 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:03.222075 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:03.225992 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967063e1-735a-4fd9-a7c7-387df0236e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:03.226918 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967063e1-735a-4fd9-a7c7-387df0236e68 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:03.227412 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967063e1-735a-4fd9-a7c7-387df0236e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:03.227886 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967063e1-735a-4fd9-a7c7-387df0236e68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:03.241061 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967063e1-735a-4fd9-a7c7-387df0236e68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:07:03.247641 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-967063e1-735a-4fd9-a7c7-387df0236e68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:07:03.247641 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:03.247641 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:03.247641 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:03.247641 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:03.247641 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:03.247641 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:03.247641 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:03.247641 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:03.247641 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:03.247641 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:03.247641 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:03.247641 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:07:03.247641 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:07:03.247641 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:07:03.247641 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:07:03.248747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:07:03.248747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:07:03.248747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:07:03.248747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:07:03.248747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:03.248747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:03.248747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:03.248747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:03.248747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:03.248747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:03.248747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:03.248747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:03.248747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:03.248747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:03.248747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:03.248747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:03.249546 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:03.249546 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:03.249546 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:03.249546 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:03.249546 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:07:03.249546 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:07:03.249546 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:07:03.249546 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:07:03.249546 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:07:03.249546 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:07:03.249546 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:07:03.249546 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:03.250945 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 216/409] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Aug 5 14:07:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:07:03.257415 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967063e1-735a-4fd9-a7c7-387df0236e68 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:03.306905 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407557e6-273e-4a8c-9d0b-365a721aacd8 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:03.310926 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:03.349593 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e3b98e-9afe-413b-8591-89f582d282f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:03.350652 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3b98e-9afe-413b-8591-89f582d282f2 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:03.351034 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3b98e-9afe-413b-8591-89f582d282f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:03.351406 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3b98e-9afe-413b-8591-89f582d282f2 None admin] PATH_INFO: `/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:03.354199 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e3b98e-9afe-413b-8591-89f582d282f2 None admin] RBAC: Authorizing `identity:get_project(project_id=ecbbd8ae8eed4ce5b441221e8e17663b)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:03.355216 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e3b98e-9afe-413b-8591-89f582d282f2 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:03.357713 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 194/410] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:07:03 2022] GET /identity/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:03.366083 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e3b98e-9afe-413b-8591-89f582d282f2 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:03.376482 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967063e1-735a-4fd9-a7c7-387df0236e68 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:03.428848 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610411ef-a5d8-4fd0-9c96-3958b030d131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:03.430498 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610411ef-a5d8-4fd0-9c96-3958b030d131 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:03.430889 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610411ef-a5d8-4fd0-9c96-3958b030d131 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:03.431268 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610411ef-a5d8-4fd0-9c96-3958b030d131 None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:03.432166 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610411ef-a5d8-4fd0-9c96-3958b030d131 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:03.432601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610411ef-a5d8-4fd0-9c96-3958b030d131 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:03.434281 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610411ef-a5d8-4fd0-9c96-3958b030d131 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:03.445104 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 217/411] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:07:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:03.451890 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610411ef-a5d8-4fd0-9c96-3958b030d131 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:03.571911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610411ef-a5d8-4fd0-9c96-3958b030d131 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:03.573686 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:03.588530 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:03.612154 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd7002-8d7b-41d0-abc4-8bf0fbdd51ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:03.613316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd7002-8d7b-41d0-abc4-8bf0fbdd51ab None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:03.613761 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd7002-8d7b-41d0-abc4-8bf0fbdd51ab None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:03.614187 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd7002-8d7b-41d0-abc4-8bf0fbdd51ab None admin] PATH_INFO: `/v3/groups/c78073879dc54a9497fd340df09dbb10` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:03.615214 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e3b98e-9afe-413b-8591-89f582d282f2 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:03.616794 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd7002-8d7b-41d0-abc4-8bf0fbdd51ab None admin] RBAC: Authorizing `identity:get_group(group_id=c78073879dc54a9497fd340df09dbb10)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:03.618413 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd7002-8d7b-41d0-abc4-8bf0fbdd51ab None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:03.620541 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 218/412] 10.222.0.26 () {56 vars in 1180 bytes} [Fri Aug 5 14:07:03 2022] GET /identity/v3/groups/c78073879dc54a9497fd340df09dbb10 => generated 229 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:03.629461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd7002-8d7b-41d0-abc4-8bf0fbdd51ab None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:03.702456 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a18bc5a-1f3c-4d44-a433-73fd685aa723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:03.705544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18bc5a-1f3c-4d44-a433-73fd685aa723 None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:03.706015 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18bc5a-1f3c-4d44-a433-73fd685aa723 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:03.707844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18bc5a-1f3c-4d44-a433-73fd685aa723 None admin] PATH_INFO: `/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b/groups/674c9fb834714b89a516d5f2cc7d5005/roles/199fd3093a824ba585b1135c8bf363be` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:03.712189 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a18bc5a-1f3c-4d44-a433-73fd685aa723 None admin] RBAC: Authorizing `identity:create_grant(project_id=ecbbd8ae8eed4ce5b441221e8e17663b, group_id=674c9fb834714b89a516d5f2cc7d5005, role_id=199fd3093a824ba585b1135c8bf363be)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:03.715820 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a18bc5a-1f3c-4d44-a433-73fd685aa723 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:03.759249 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd7002-8d7b-41d0-abc4-8bf0fbdd51ab None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:03.797245 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 195/413] 10.222.0.26 () {58 vars in 1442 bytes} [Fri Aug 5 14:07:03 2022] PUT /identity/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b/groups/674c9fb834714b89a516d5f2cc7d5005/roles/199fd3093a824ba585b1135c8bf363be => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:07:03.808925 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d81b64-7c5a-444f-8ce5-da2f090b0135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:03.809928 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d81b64-7c5a-444f-8ce5-da2f090b0135 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:03.811317 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d81b64-7c5a-444f-8ce5-da2f090b0135 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:03.811317 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d81b64-7c5a-444f-8ce5-da2f090b0135 None admin] PATH_INFO: `/v3/projects/b40e0244e98a49699194b094ad410d1e` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:03.812316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d81b64-7c5a-444f-8ce5-da2f090b0135 None admin] RBAC: Authorizing `identity:get_project(project_id=b40e0244e98a49699194b094ad410d1e)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:03.814182 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d81b64-7c5a-444f-8ce5-da2f090b0135 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:03.816588 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 219/414] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:07:03 2022] GET /identity/v3/projects/b40e0244e98a49699194b094ad410d1e => generated 344 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:03.823519 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a18bc5a-1f3c-4d44-a433-73fd685aa723 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:03.940917 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a18bc5a-1f3c-4d44-a433-73fd685aa723 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:03.980222 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277fe842-d49f-4c95-a343-5064626e1928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:03.981188 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277fe842-d49f-4c95-a343-5064626e1928 None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:03.981634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277fe842-d49f-4c95-a343-5064626e1928 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:03.982079 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277fe842-d49f-4c95-a343-5064626e1928 None admin] PATH_INFO: `/v3/projects/b40e0244e98a49699194b094ad410d1e/groups/c78073879dc54a9497fd340df09dbb10/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:03.984264 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277fe842-d49f-4c95-a343-5064626e1928 None admin] RBAC: Authorizing `identity:create_grant(project_id=b40e0244e98a49699194b094ad410d1e, group_id=c78073879dc54a9497fd340df09dbb10, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:03.985979 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277fe842-d49f-4c95-a343-5064626e1928 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:04.068624 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 196/415] 10.222.0.26 () {58 vars in 1442 bytes} [Fri Aug 5 14:07:03 2022] PUT /identity/v3/projects/b40e0244e98a49699194b094ad410d1e/groups/c78073879dc54a9497fd340df09dbb10/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:07:04.134996 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453a6ab-9fac-48cb-8fda-c3f608ab6898 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:04.135323 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453a6ab-9fac-48cb-8fda-c3f608ab6898 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:04.135551 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453a6ab-9fac-48cb-8fda-c3f608ab6898 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:04.137133 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 220/416] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:04.143335 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acfc52-9b6e-443c-8ae3-a19520699a64 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:04.143624 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acfc52-9b6e-443c-8ae3-a19520699a64 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:04.143896 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acfc52-9b6e-443c-8ae3-a19520699a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:04.170081 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-b6acfc52-9b6e-443c-8ae3-a19520699a64 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:04.171383 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-b6acfc52-9b6e-443c-8ae3-a19520699a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:04.180243 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:04.265139 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 197/417] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:07:04.301999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9453a6ab-9fac-48cb-8fda-c3f608ab6898 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:04.356395 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:04.446086 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9453a6ab-9fac-48cb-8fda-c3f608ab6898 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:04.493935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954bcbe4-dd6c-4b05-8c60-81b71f2596d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:04.494591 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954bcbe4-dd6c-4b05-8c60-81b71f2596d5 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:04.494870 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954bcbe4-dd6c-4b05-8c60-81b71f2596d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:04.495140 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954bcbe4-dd6c-4b05-8c60-81b71f2596d5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:04.517703 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954bcbe4-dd6c-4b05-8c60-81b71f2596d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:07:04.524999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-954bcbe4-dd6c-4b05-8c60-81b71f2596d5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 14:07:04.524999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:04.524999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:04.524999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:04.524999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:04.524999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:04.524999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:04.524999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:04.524999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:04.524999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:04.524999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:04.524999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:04.524999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:07:04.524999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:07:04.524999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:07:04.524999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:07:04.525935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:07:04.525935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:07:04.525935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:07:04.525935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:07:04.525935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:04.525935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:04.525935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:04.525935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:04.525935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:04.525935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:04.525935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:04.525935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:04.525935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:04.525935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:04.525935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:04.525935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:04.526720 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:04.526720 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:04.526720 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:04.526720 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:04.526720 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:07:04.526720 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:07:04.526720 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:07:04.526720 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:07:04.526720 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:07:04.526720 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:07:04.526720 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 14:07:04.526720 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:04.526720 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 221/418] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:07:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:04.532207 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6acfc52-9b6e-443c-8ae3-a19520699a64 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:04.686947 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6acfc52-9b6e-443c-8ae3-a19520699a64 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:04.737668 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66a967-7845-4e54-b615-cadc51ae0827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:04.739088 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66a967-7845-4e54-b615-cadc51ae0827 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:04.740499 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66a967-7845-4e54-b615-cadc51ae0827 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:04.741053 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66a967-7845-4e54-b615-cadc51ae0827 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:04.742068 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c66a967-7845-4e54-b615-cadc51ae0827 None admin] RBAC: Adding query filter params (name=member) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:04.742561 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c66a967-7845-4e54-b615-cadc51ae0827 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:04.744177 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c66a967-7845-4e54-b615-cadc51ae0827 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:04.761583 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 198/419] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Aug 5 14:07:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:04.766833 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954bcbe4-dd6c-4b05-8c60-81b71f2596d5 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:04.897377 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:04.900845 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954bcbe4-dd6c-4b05-8c60-81b71f2596d5 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:04.950940 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743fd81-a885-4ec4-88e7-dcacc7e9ee27 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:04.951567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743fd81-a885-4ec4-88e7-dcacc7e9ee27 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:04.952042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743fd81-a885-4ec4-88e7-dcacc7e9ee27 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:04.953734 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 199/420] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:07:04.957788 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306ebb8-ec02-49aa-993a-9e5a1c10121a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:04.962827 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306ebb8-ec02-49aa-993a-9e5a1c10121a None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:04.962827 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306ebb8-ec02-49aa-993a-9e5a1c10121a None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:04.962827 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306ebb8-ec02-49aa-993a-9e5a1c10121a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:04.964596 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c87348-9fc2-4c68-873b-0c142cd2dce4 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:04.965103 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c87348-9fc2-4c68-873b-0c142cd2dce4 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:04.965643 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c87348-9fc2-4c68-873b-0c142cd2dce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:04.975270 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:04.996115 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-b1c87348-9fc2-4c68-873b-0c142cd2dce4 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:05.000372 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-b1c87348-9fc2-4c68-873b-0c142cd2dce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:05.061550 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5306ebb8-ec02-49aa-993a-9e5a1c10121a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:05.063336 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5306ebb8-ec02-49aa-993a-9e5a1c10121a None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:05.125820 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 200/421] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:07:05.172959 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:05.177976 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c87348-9fc2-4c68-873b-0c142cd2dce4 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:05.187476 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 222/422] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:07:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 14:07:05.197275 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306ebb8-ec02-49aa-993a-9e5a1c10121a None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:05.339010 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c87348-9fc2-4c68-873b-0c142cd2dce4 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:05.376065 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d84a57-3dc4-43e3-8b3c-bf940cf16a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:05.376747 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d84a57-3dc4-43e3-8b3c-bf940cf16a55 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:05.377023 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d84a57-3dc4-43e3-8b3c-bf940cf16a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:05.377294 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d84a57-3dc4-43e3-8b3c-bf940cf16a55 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:05.379320 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:05.382480 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306ebb8-ec02-49aa-993a-9e5a1c10121a None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:05.390685 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d84a57-3dc4-43e3-8b3c-bf940cf16a55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:07:05.396599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-b5d84a57-3dc4-43e3-8b3c-bf940cf16a55 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 14:07:05.396599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:05.396599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:05.396599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:05.396599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:05.396599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:05.396599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:05.396599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:05.396599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:05.396599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:05.396599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:05.396599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:05.396599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:07:05.396599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:07:05.396599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:07:05.396599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:07:05.397647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:07:05.397647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:07:05.397647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:07:05.397647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:07:05.397647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:05.397647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:05.397647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:05.397647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:05.397647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:05.397647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:05.397647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:05.397647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:05.397647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:05.397647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:05.397647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:05.397647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:05.398457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:05.398457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:05.398457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:05.398457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:05.398457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:07:05.398457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:07:05.398457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:07:05.398457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:07:05.398457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:07:05.398457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:07:05.398457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 14:07:05.398457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:05.398457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 201/423] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Aug 5 14:07:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:05.399786 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40070d1a-3e4a-4271-9cb1-6bb0984f6ad6 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:05.400558 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40070d1a-3e4a-4271-9cb1-6bb0984f6ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:05.404953 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40070d1a-3e4a-4271-9cb1-6bb0984f6ad6 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:05.406683 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 202/424] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:05.409116 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40070d1a-3e4a-4271-9cb1-6bb0984f6ad6 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:05.430505 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6826cdd1-c661-42f8-b308-cb261ebf7959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:05.431079 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826cdd1-c661-42f8-b308-cb261ebf7959 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:05.431356 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826cdd1-c661-42f8-b308-cb261ebf7959 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:05.431629 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826cdd1-c661-42f8-b308-cb261ebf7959 None admin] PATH_INFO: `/v3/projects/580a155c67b742b9bc398f50556f26cf` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:05.433161 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6826cdd1-c661-42f8-b308-cb261ebf7959 None admin] RBAC: Authorizing `identity:get_project(project_id=580a155c67b742b9bc398f50556f26cf)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:05.434622 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6826cdd1-c661-42f8-b308-cb261ebf7959 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:05.437491 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 223/425] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:07:05 2022] GET /identity/v3/projects/580a155c67b742b9bc398f50556f26cf => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:05.442127 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9ab31-9495-453f-bae5-e8dbd3e68dd9 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:05.442127 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9ab31-9495-453f-bae5-e8dbd3e68dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:05.442127 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9ab31-9495-453f-bae5-e8dbd3e68dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:05.464862 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-92c9ab31-9495-453f-bae5-e8dbd3e68dd9 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:05.466110 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-92c9ab31-9495-453f-bae5-e8dbd3e68dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:05.533583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:05.548583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 224/426] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:07:05.548583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9ab31-9495-453f-bae5-e8dbd3e68dd9 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:05.566726 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40070d1a-3e4a-4271-9cb1-6bb0984f6ad6 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:05.600338 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee670150-bfcd-472e-ab01-0242b9298a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:05.600608 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee670150-bfcd-472e-ab01-0242b9298a6e None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:05.600937 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee670150-bfcd-472e-ab01-0242b9298a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:05.601163 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee670150-bfcd-472e-ab01-0242b9298a6e None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:05.601905 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee670150-bfcd-472e-ab01-0242b9298a6e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:05.602214 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee670150-bfcd-472e-ab01-0242b9298a6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:05.603711 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee670150-bfcd-472e-ab01-0242b9298a6e None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:05.613419 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 203/427] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:07:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:05.614815 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee670150-bfcd-472e-ab01-0242b9298a6e None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:05.666738 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9ab31-9495-453f-bae5-e8dbd3e68dd9 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:05.679300 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:05.703479 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c696e9-c113-4bed-b800-93357ff20f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:05.704478 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c696e9-c113-4bed-b800-93357ff20f72 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:05.704986 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c696e9-c113-4bed-b800-93357ff20f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:05.705466 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c696e9-c113-4bed-b800-93357ff20f72 None admin] PATH_INFO: `/v3/groups/674c9fb834714b89a516d5f2cc7d5005` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:05.706814 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c696e9-c113-4bed-b800-93357ff20f72 None admin] RBAC: Authorizing `identity:get_group(group_id=674c9fb834714b89a516d5f2cc7d5005)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:05.708796 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c696e9-c113-4bed-b800-93357ff20f72 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:05.715476 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 225/428] 10.222.0.26 () {56 vars in 1180 bytes} [Fri Aug 5 14:07:05 2022] GET /identity/v3/groups/674c9fb834714b89a516d5f2cc7d5005 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:05.716716 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c696e9-c113-4bed-b800-93357ff20f72 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:05.727311 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee670150-bfcd-472e-ab01-0242b9298a6e None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:05.763853 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f9732-8670-4bd0-89ac-a0cf3099f8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:05.764606 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f9732-8670-4bd0-89ac-a0cf3099f8c9 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:05.764796 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f9732-8670-4bd0-89ac-a0cf3099f8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:05.765070 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f9732-8670-4bd0-89ac-a0cf3099f8c9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:05.776199 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989f9732-8670-4bd0-89ac-a0cf3099f8c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:07:05.781804 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-989f9732-8670-4bd0-89ac-a0cf3099f8c9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 14:07:05.781804 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:05.781804 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:05.781804 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:05.781804 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:05.781804 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:05.781804 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:05.781804 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:05.781804 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:05.781804 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:05.781804 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:05.781804 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:05.781804 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:07:05.781804 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:07:05.781804 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:07:05.781804 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:07:05.782735 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:07:05.782735 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:07:05.782735 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:07:05.782735 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:07:05.782735 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:05.782735 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:05.782735 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:05.782735 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:05.782735 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:05.782735 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:05.782735 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:05.782735 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:05.782735 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:05.782735 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:05.782735 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:05.782735 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:05.783515 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:05.783515 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:05.783515 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:05.783515 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:05.783515 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:07:05.783515 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:07:05.783515 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:07:05.783515 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:07:05.783515 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:07:05.783515 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:07:05.783515 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 14:07:05.783515 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:05.783515 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 204/429] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:07:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:05.784174 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f9732-8670-4bd0-89ac-a0cf3099f8c9 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:05.787425 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:05.827410 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c696e9-c113-4bed-b800-93357ff20f72 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:05.863031 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae4a297-9445-42f9-9b2e-6b1f37d14f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:05.863998 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4a297-9445-42f9-9b2e-6b1f37d14f1a None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:05.864663 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4a297-9445-42f9-9b2e-6b1f37d14f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:05.865144 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4a297-9445-42f9-9b2e-6b1f37d14f1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:05.897186 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f9732-8670-4bd0-89ac-a0cf3099f8c9 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:05.914762 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:05.934168 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e66495-1302-469c-b1c5-611a0d497bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:05.934797 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e66495-1302-469c-b1c5-611a0d497bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:05.935081 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e66495-1302-469c-b1c5-611a0d497bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:05.935411 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e66495-1302-469c-b1c5-611a0d497bf3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:05.935897 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae4a297-9445-42f9-9b2e-6b1f37d14f1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:05.936352 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e66495-1302-469c-b1c5-611a0d497bf3 None admin] RBAC: Adding query filter params (group.id=674c9fb834714b89a516d5f2cc7d5005, role.id=29967df108484a30a4c006cde2ae2940, scope.project.id=580a155c67b742b9bc398f50556f26cf) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:05.936736 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e66495-1302-469c-b1c5-611a0d497bf3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:05.938046 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae4a297-9445-42f9-9b2e-6b1f37d14f1a None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:05.938160 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e66495-1302-469c-b1c5-611a0d497bf3 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:05.950106 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 205/430] 10.222.0.26 () {56 vars in 1376 bytes} [Fri Aug 5 14:07:05 2022] GET /identity/v3/role_assignments?group.id=674c9fb834714b89a516d5f2cc7d5005&scope.project.id=580a155c67b742b9bc398f50556f26cf&role.id=29967df108484a30a4c006cde2ae2940 => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:05.950106 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e66495-1302-469c-b1c5-611a0d497bf3 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:06.064916 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:06.065532 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e66495-1302-469c-b1c5-611a0d497bf3 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:06.104274 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c242d650-ce73-48fa-a538-c67755b6f3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:06.105002 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c242d650-ce73-48fa-a538-c67755b6f3f9 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:06.105282 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c242d650-ce73-48fa-a538-c67755b6f3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:06.105554 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c242d650-ce73-48fa-a538-c67755b6f3f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:06.106285 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c242d650-ce73-48fa-a538-c67755b6f3f9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:06.106591 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c242d650-ce73-48fa-a538-c67755b6f3f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:06.107995 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c242d650-ce73-48fa-a538-c67755b6f3f9 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:06.117571 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 206/431] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Aug 5 14:07:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:06.123564 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c242d650-ce73-48fa-a538-c67755b6f3f9 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:06.125143 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 226/432] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:07:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 14:07:06.137521 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae4a297-9445-42f9-9b2e-6b1f37d14f1a None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:06.178317 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:06.243663 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c242d650-ce73-48fa-a538-c67755b6f3f9 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:06.283974 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b68d2e-0988-4398-a735-7c0f96bfec34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:06.284958 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b68d2e-0988-4398-a735-7c0f96bfec34 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:06.285509 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b68d2e-0988-4398-a735-7c0f96bfec34 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:06.285787 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b68d2e-0988-4398-a735-7c0f96bfec34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:06.290440 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae4a297-9445-42f9-9b2e-6b1f37d14f1a None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:06.338916 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc952545-639d-40c2-9c5e-b283308bf8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:06.339491 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc952545-639d-40c2-9c5e-b283308bf8df None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:06.339768 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc952545-639d-40c2-9c5e-b283308bf8df None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:06.340038 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc952545-639d-40c2-9c5e-b283308bf8df None admin] PATH_INFO: `/v3/projects/580a155c67b742b9bc398f50556f26cf` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:06.343396 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc952545-639d-40c2-9c5e-b283308bf8df None admin] RBAC: Authorizing `identity:get_project(project_id=580a155c67b742b9bc398f50556f26cf)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:06.346037 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc952545-639d-40c2-9c5e-b283308bf8df None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:06.347949 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 227/433] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:07:06 2022] GET /identity/v3/projects/580a155c67b742b9bc398f50556f26cf => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:06.349718 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aaa3f1-fdbd-4a49-9bf7-fa7fbe9e0064 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:06.349718 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aaa3f1-fdbd-4a49-9bf7-fa7fbe9e0064 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:06.349718 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aaa3f1-fdbd-4a49-9bf7-fa7fbe9e0064 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:06.353374 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 228/434] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:06.355722 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aaa3f1-fdbd-4a49-9bf7-fa7fbe9e0064 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:06.375402 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b68d2e-0988-4398-a735-7c0f96bfec34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:06.377064 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b68d2e-0988-4398-a735-7c0f96bfec34 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:06.393093 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:06.425320 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:06.478347 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aaa3f1-fdbd-4a49-9bf7-fa7fbe9e0064 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:06.503907 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 207/435] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:07:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 14:07:06.507147 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052229a3-5a7e-4e38-959c-37ba3c67fbbf None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:06.507431 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052229a3-5a7e-4e38-959c-37ba3c67fbbf None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:06.507706 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052229a3-5a7e-4e38-959c-37ba3c67fbbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:06.524447 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bbfa53-786b-450f-ab5c-beefe6f16526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:06.526152 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbfa53-786b-450f-ab5c-beefe6f16526 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:06.526637 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbfa53-786b-450f-ab5c-beefe6f16526 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:06.527069 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbfa53-786b-450f-ab5c-beefe6f16526 None admin] PATH_INFO: `/v3/groups/674c9fb834714b89a516d5f2cc7d5005` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:06.528312 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bbfa53-786b-450f-ab5c-beefe6f16526 None admin] RBAC: Authorizing `identity:get_group(group_id=674c9fb834714b89a516d5f2cc7d5005)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:06.529963 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bbfa53-786b-450f-ab5c-beefe6f16526 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:06.531576 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 229/436] 10.222.0.26 () {56 vars in 1180 bytes} [Fri Aug 5 14:07:06 2022] GET /identity/v3/groups/674c9fb834714b89a516d5f2cc7d5005 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:06.532336 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bbfa53-786b-450f-ab5c-beefe6f16526 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:06.532988 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-052229a3-5a7e-4e38-959c-37ba3c67fbbf None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:06.534515 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-052229a3-5a7e-4e38-959c-37ba3c67fbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:06.612233 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 208/437] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:07:06.613204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052229a3-5a7e-4e38-959c-37ba3c67fbbf None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:06.658715 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:06.662144 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bbfa53-786b-450f-ab5c-beefe6f16526 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:06.693877 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:06.696978 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafefb75-fd5a-4817-86f0-5301a563f414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:06.697615 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafefb75-fd5a-4817-86f0-5301a563f414 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:06.697887 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafefb75-fd5a-4817-86f0-5301a563f414 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:06.698195 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafefb75-fd5a-4817-86f0-5301a563f414 None admin] PATH_INFO: `/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:06.699348 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafefb75-fd5a-4817-86f0-5301a563f414 None admin] RBAC: Authorizing `identity:get_project(project_id=ecbbd8ae8eed4ce5b441221e8e17663b)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:06.700825 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafefb75-fd5a-4817-86f0-5301a563f414 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:06.706088 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 230/438] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:07:06 2022] GET /identity/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:06.706088 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83b2e7-3e9d-489f-b02a-364db4b022dd None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:06.706088 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83b2e7-3e9d-489f-b02a-364db4b022dd None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:06.706088 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83b2e7-3e9d-489f-b02a-364db4b022dd None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:06.706088 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 231/439] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:06.706088 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83b2e7-3e9d-489f-b02a-364db4b022dd None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:06.728851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052229a3-5a7e-4e38-959c-37ba3c67fbbf None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:06.762613 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83344760-9acd-4ac9-bded-edbcc7448f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:06.763219 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83344760-9acd-4ac9-bded-edbcc7448f1e None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:06.763492 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83344760-9acd-4ac9-bded-edbcc7448f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:06.763758 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83344760-9acd-4ac9-bded-edbcc7448f1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:06.764568 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83344760-9acd-4ac9-bded-edbcc7448f1e None admin] RBAC: Adding query filter params (group.id=674c9fb834714b89a516d5f2cc7d5005, role.id=199fd3093a824ba585b1135c8bf363be, scope.project.id=580a155c67b742b9bc398f50556f26cf) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:06.765096 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83344760-9acd-4ac9-bded-edbcc7448f1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:06.766270 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83344760-9acd-4ac9-bded-edbcc7448f1e None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:06.772213 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 209/440] 10.222.0.26 () {56 vars in 1376 bytes} [Fri Aug 5 14:07:06 2022] GET /identity/v3/role_assignments?group.id=674c9fb834714b89a516d5f2cc7d5005&scope.project.id=580a155c67b742b9bc398f50556f26cf&role.id=199fd3093a824ba585b1135c8bf363be => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:06.773076 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83344760-9acd-4ac9-bded-edbcc7448f1e None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:06.830653 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83b2e7-3e9d-489f-b02a-364db4b022dd None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:06.870494 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dfa04a-ce72-4600-804f-d1f5c99b988f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:06.871056 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfa04a-ce72-4600-804f-d1f5c99b988f None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:06.871336 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfa04a-ce72-4600-804f-d1f5c99b988f None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:06.871600 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfa04a-ce72-4600-804f-d1f5c99b988f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:06.884393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dfa04a-ce72-4600-804f-d1f5c99b988f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:07:06.886920 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83344760-9acd-4ac9-bded-edbcc7448f1e None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:06.889420 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-19dfa04a-ce72-4600-804f-d1f5c99b988f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 14:07:06.889420 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:06.889420 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:06.889420 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:06.889420 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:06.889420 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:06.889420 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:06.889420 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:06.889420 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:06.889420 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:06.889420 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:06.889420 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:06.889420 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:07:06.889420 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:07:06.889420 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:07:06.889420 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:07:06.890387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:07:06.890387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:07:06.890387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:07:06.890387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:07:06.890387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:06.890387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:06.890387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:06.890387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:06.890387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:06.890387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:06.890387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:06.890387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:06.890387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:06.890387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:06.890387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:06.890387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:06.891288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:06.891288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:06.891288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:06.891288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:06.891288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:07:06.891288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:07:06.891288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:07:06.891288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:07:06.891288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:07:06.891288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:07:06.891288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 14:07:06.891288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:06.891288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 232/441] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Aug 5 14:07:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:06.891967 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e93a8f-f389-4e18-af14-d99228c4e9ee None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:06.892404 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e93a8f-f389-4e18-af14-d99228c4e9ee None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:06.892530 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e93a8f-f389-4e18-af14-d99228c4e9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:06.912688 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-11e93a8f-f389-4e18-af14-d99228c4e9ee None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:06.913929 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-11e93a8f-f389-4e18-af14-d99228c4e9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:06.934209 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7226d7b-855e-4abe-935b-8f19d8c53e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:06.934209 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7226d7b-855e-4abe-935b-8f19d8c53e7b None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:06.934498 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7226d7b-855e-4abe-935b-8f19d8c53e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:06.934669 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7226d7b-855e-4abe-935b-8f19d8c53e7b None admin] PATH_INFO: `/v3/groups/674c9fb834714b89a516d5f2cc7d5005` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:06.935762 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7226d7b-855e-4abe-935b-8f19d8c53e7b None admin] RBAC: Authorizing `identity:get_group(group_id=674c9fb834714b89a516d5f2cc7d5005)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:06.937213 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7226d7b-855e-4abe-935b-8f19d8c53e7b None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:06.939091 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 210/442] 10.222.0.26 () {56 vars in 1180 bytes} [Fri Aug 5 14:07:06 2022] GET /identity/v3/groups/674c9fb834714b89a516d5f2cc7d5005 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:06.939597 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7226d7b-855e-4abe-935b-8f19d8c53e7b None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:06.944724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:06.947437 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:06.998409 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 233/443] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:07:06.999076 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e93a8f-f389-4e18-af14-d99228c4e9ee None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:07.066306 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7226d7b-855e-4abe-935b-8f19d8c53e7b None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:07.110225 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adccc03-d5a4-4311-a24e-3863d990096a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:07.111008 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adccc03-d5a4-4311-a24e-3863d990096a None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:07.111293 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adccc03-d5a4-4311-a24e-3863d990096a None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:07.111885 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adccc03-d5a4-4311-a24e-3863d990096a None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:07.112650 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adccc03-d5a4-4311-a24e-3863d990096a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:07.112960 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adccc03-d5a4-4311-a24e-3863d990096a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:07.114966 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adccc03-d5a4-4311-a24e-3863d990096a None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:07.121081 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e93a8f-f389-4e18-af14-d99228c4e9ee None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:07.123469 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 211/444] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:07:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:07.124273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adccc03-d5a4-4311-a24e-3863d990096a None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:07.158687 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e820b-106b-4165-b206-4920dc95bb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:07.159311 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e820b-106b-4165-b206-4920dc95bb5c None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:07.159587 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e820b-106b-4165-b206-4920dc95bb5c None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:07.159857 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e820b-106b-4165-b206-4920dc95bb5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:07.160681 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e820b-106b-4165-b206-4920dc95bb5c None admin] RBAC: Adding query filter params (group.id=674c9fb834714b89a516d5f2cc7d5005, role.id=29967df108484a30a4c006cde2ae2940, scope.project.id=ecbbd8ae8eed4ce5b441221e8e17663b) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:07.161563 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e820b-106b-4165-b206-4920dc95bb5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:07.162984 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e820b-106b-4165-b206-4920dc95bb5c None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:07.168728 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 234/445] 10.222.0.26 () {56 vars in 1376 bytes} [Fri Aug 5 14:07:06 2022] GET /identity/v3/role_assignments?group.id=674c9fb834714b89a516d5f2cc7d5005&scope.project.id=ecbbd8ae8eed4ce5b441221e8e17663b&role.id=29967df108484a30a4c006cde2ae2940 => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:07.169509 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e820b-106b-4165-b206-4920dc95bb5c None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:07.197335 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:07.206108 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:07.234346 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adccc03-d5a4-4311-a24e-3863d990096a None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:07.270876 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95dd0b3-6921-4875-80b7-5124f8c2b8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:07.271957 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95dd0b3-6921-4875-80b7-5124f8c2b8bf None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:07.272611 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95dd0b3-6921-4875-80b7-5124f8c2b8bf None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:07.273482 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95dd0b3-6921-4875-80b7-5124f8c2b8bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:07.285043 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95dd0b3-6921-4875-80b7-5124f8c2b8bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:07:07.288971 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e820b-106b-4165-b206-4920dc95bb5c None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:07.291749 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-e95dd0b3-6921-4875-80b7-5124f8c2b8bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:07:07.291749 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:07.291749 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:07.291749 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:07.291749 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:07.291749 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:07.291749 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:07.291749 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:07.291749 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:07.291749 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:07.291749 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:07.291749 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:07.291749 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:07:07.291749 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:07:07.291749 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:07:07.291749 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:07:07.292844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:07:07.292844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:07:07.292844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:07:07.292844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:07:07.292844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:07.292844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:07.292844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:07.292844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:07.292844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:07.292844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:07.292844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:07.292844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:07.292844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:07.292844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:07.292844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:07.292844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:07.293814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:07.293814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:07.293814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:07.293814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:07.293814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:07:07.293814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:07:07.293814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:07:07.293814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:07:07.293814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:07:07.293814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:07:07.293814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:07:07.293814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:07.293814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 212/446] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Aug 5 14:07:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:07.299262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95dd0b3-6921-4875-80b7-5124f8c2b8bf None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:07.333704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b59277-c19b-41c9-b82b-6c446ce74eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:07.334451 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b59277-c19b-41c9-b82b-6c446ce74eff None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:07.334726 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b59277-c19b-41c9-b82b-6c446ce74eff None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:07.335031 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b59277-c19b-41c9-b82b-6c446ce74eff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:07.414363 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b59277-c19b-41c9-b82b-6c446ce74eff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:07.415847 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b59277-c19b-41c9-b82b-6c446ce74eff None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:07.416028 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95dd0b3-6921-4875-80b7-5124f8c2b8bf None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:07.441260 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:07.451427 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:07.462010 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc1e209-8d4b-4445-81d2-a2fe8c25a943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:07.462626 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1e209-8d4b-4445-81d2-a2fe8c25a943 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:07.462901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1e209-8d4b-4445-81d2-a2fe8c25a943 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:07.463168 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1e209-8d4b-4445-81d2-a2fe8c25a943 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:07.463882 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc1e209-8d4b-4445-81d2-a2fe8c25a943 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:07.464182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc1e209-8d4b-4445-81d2-a2fe8c25a943 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:07.465630 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc1e209-8d4b-4445-81d2-a2fe8c25a943 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:07.475883 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 213/447] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:07:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:07.481398 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc1e209-8d4b-4445-81d2-a2fe8c25a943 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:07.536693 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 235/448] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:07:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 14:07:07.542243 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b59277-c19b-41c9-b82b-6c446ce74eff None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:07.596672 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc1e209-8d4b-4445-81d2-a2fe8c25a943 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:07.631007 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307b1ad8-a03c-4269-b07f-b3adc23a3b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:07.631628 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b1ad8-a03c-4269-b07f-b3adc23a3b8e None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:07.631915 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b1ad8-a03c-4269-b07f-b3adc23a3b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:07.632184 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b1ad8-a03c-4269-b07f-b3adc23a3b8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:07.639837 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b59277-c19b-41c9-b82b-6c446ce74eff None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:07.663230 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:07.684524 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e23724-d0e9-437c-adb0-20c5e2ec45c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:07.684524 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e23724-d0e9-437c-adb0-20c5e2ec45c6 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:07.684524 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e23724-d0e9-437c-adb0-20c5e2ec45c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:07.684524 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e23724-d0e9-437c-adb0-20c5e2ec45c6 None admin] PATH_INFO: `/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:07.684910 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e23724-d0e9-437c-adb0-20c5e2ec45c6 None admin] RBAC: Authorizing `identity:get_project(project_id=ecbbd8ae8eed4ce5b441221e8e17663b)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:07.684910 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e23724-d0e9-437c-adb0-20c5e2ec45c6 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:07.684910 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 236/449] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:07:07 2022] GET /identity/v3/projects/ecbbd8ae8eed4ce5b441221e8e17663b => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:07.685089 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e23724-d0e9-437c-adb0-20c5e2ec45c6 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:07.687040 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:07.698757 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307b1ad8-a03c-4269-b07f-b3adc23a3b8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:07.700193 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307b1ad8-a03c-4269-b07f-b3adc23a3b8e None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:07.784645 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e23724-d0e9-437c-adb0-20c5e2ec45c6 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:07.799442 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 214/450] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:07:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 14:07:07.806011 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307b1ad8-a03c-4269-b07f-b3adc23a3b8e None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:07.818367 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648c3736-d876-4e0c-8618-2cb37ae42838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:07.819003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c3736-d876-4e0c-8618-2cb37ae42838 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:07.819280 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c3736-d876-4e0c-8618-2cb37ae42838 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:07.819552 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c3736-d876-4e0c-8618-2cb37ae42838 None admin] PATH_INFO: `/v3/groups/674c9fb834714b89a516d5f2cc7d5005` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:07.820734 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648c3736-d876-4e0c-8618-2cb37ae42838 None admin] RBAC: Authorizing `identity:get_group(group_id=674c9fb834714b89a516d5f2cc7d5005)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:07.822173 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648c3736-d876-4e0c-8618-2cb37ae42838 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:07.823723 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 237/451] 10.222.0.26 () {56 vars in 1180 bytes} [Fri Aug 5 14:07:07 2022] GET /identity/v3/groups/674c9fb834714b89a516d5f2cc7d5005 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:07.829317 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648c3736-d876-4e0c-8618-2cb37ae42838 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:07.888144 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:07.900001 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:07.911760 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307b1ad8-a03c-4269-b07f-b3adc23a3b8e None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:07.931124 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648c3736-d876-4e0c-8618-2cb37ae42838 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:07.945744 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c097fa-d9dc-478f-868d-e076d4687070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:07.946306 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c097fa-d9dc-478f-868d-e076d4687070 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:07.946579 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c097fa-d9dc-478f-868d-e076d4687070 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:07.946847 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c097fa-d9dc-478f-868d-e076d4687070 None admin] PATH_INFO: `/v3/projects/b40e0244e98a49699194b094ad410d1e` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:07.947976 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c097fa-d9dc-478f-868d-e076d4687070 None admin] RBAC: Authorizing `identity:get_project(project_id=b40e0244e98a49699194b094ad410d1e)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:07.949429 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c097fa-d9dc-478f-868d-e076d4687070 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:07.951041 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 215/452] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:07:07 2022] GET /identity/v3/projects/b40e0244e98a49699194b094ad410d1e => generated 344 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:07.956523 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c097fa-d9dc-478f-868d-e076d4687070 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:07.966360 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2559b93-fe47-4fe8-b58f-a5ba1ed01f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:07.966977 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2559b93-fe47-4fe8-b58f-a5ba1ed01f9a None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:07.967255 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2559b93-fe47-4fe8-b58f-a5ba1ed01f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:07.967524 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2559b93-fe47-4fe8-b58f-a5ba1ed01f9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:07.968306 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2559b93-fe47-4fe8-b58f-a5ba1ed01f9a None admin] RBAC: Adding query filter params (group.id=674c9fb834714b89a516d5f2cc7d5005, role.id=199fd3093a824ba585b1135c8bf363be, scope.project.id=ecbbd8ae8eed4ce5b441221e8e17663b) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:07.968648 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2559b93-fe47-4fe8-b58f-a5ba1ed01f9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:07.970054 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2559b93-fe47-4fe8-b58f-a5ba1ed01f9a None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:07.975887 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 238/453] 10.222.0.26 () {56 vars in 1376 bytes} [Fri Aug 5 14:07:07 2022] GET /identity/v3/role_assignments?group.id=674c9fb834714b89a516d5f2cc7d5005&scope.project.id=ecbbd8ae8eed4ce5b441221e8e17663b&role.id=199fd3093a824ba585b1135c8bf363be => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:08.058008 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c097fa-d9dc-478f-868d-e076d4687070 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:08.092935 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c12715-c8ca-463a-94d7-f9d92fc8e3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:08.093583 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c12715-c8ca-463a-94d7-f9d92fc8e3c9 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:08.093906 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c12715-c8ca-463a-94d7-f9d92fc8e3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:08.094176 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c12715-c8ca-463a-94d7-f9d92fc8e3c9 None admin] PATH_INFO: `/v3/groups/c78073879dc54a9497fd340df09dbb10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:08.095331 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c12715-c8ca-463a-94d7-f9d92fc8e3c9 None admin] RBAC: Authorizing `identity:get_group(group_id=c78073879dc54a9497fd340df09dbb10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:08.096787 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c12715-c8ca-463a-94d7-f9d92fc8e3c9 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:08.098386 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 216/454] 10.222.0.26 () {56 vars in 1180 bytes} [Fri Aug 5 14:07:07 2022] GET /identity/v3/groups/c78073879dc54a9497fd340df09dbb10 => generated 229 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:08.104515 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2559b93-fe47-4fe8-b58f-a5ba1ed01f9a None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:08.206862 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2559b93-fe47-4fe8-b58f-a5ba1ed01f9a None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:08.229455 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:08.242289 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3527cda2-1698-4c71-bafc-f4732b39256b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:08.249175 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527cda2-1698-4c71-bafc-f4732b39256b None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:08.249175 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527cda2-1698-4c71-bafc-f4732b39256b None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:08.249175 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527cda2-1698-4c71-bafc-f4732b39256b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:08.249175 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3527cda2-1698-4c71-bafc-f4732b39256b None admin] RBAC: Adding query filter params (group.id=c78073879dc54a9497fd340df09dbb10, role.id=299c794c7acb4a5ca36288c3191aad10, scope.project.id=b40e0244e98a49699194b094ad410d1e) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:08.249175 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3527cda2-1698-4c71-bafc-f4732b39256b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:08.249175 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3527cda2-1698-4c71-bafc-f4732b39256b None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:08.252472 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 239/455] 10.222.0.26 () {56 vars in 1376 bytes} [Fri Aug 5 14:07:08 2022] GET /identity/v3/role_assignments?group.id=c78073879dc54a9497fd340df09dbb10&scope.project.id=b40e0244e98a49699194b094ad410d1e&role.id=299c794c7acb4a5ca36288c3191aad10 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:10.805280 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609e5a0-2fb7-492d-9126-936dd5f5b983 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:10.805792 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609e5a0-2fb7-492d-9126-936dd5f5b983 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:10.805792 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609e5a0-2fb7-492d-9126-936dd5f5b983 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:10.807071 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 217/456] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:10.812469 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9294fc-37ed-426c-bf44-309d7d4edb90 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:10.813280 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9294fc-37ed-426c-bf44-309d7d4edb90 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:10.813777 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9294fc-37ed-426c-bf44-309d7d4edb90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:10.833311 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-6b9294fc-37ed-426c-bf44-309d7d4edb90 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:10.834817 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-6b9294fc-37ed-426c-bf44-309d7d4edb90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:10.875904 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af699df2-7eb2-4c4f-a6e6-d5779129f45b None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:10.876567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af699df2-7eb2-4c4f-a6e6-d5779129f45b None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:10.877067 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af699df2-7eb2-4c4f-a6e6-d5779129f45b None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:10.878836 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 218/457] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:10.884763 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f79e8-b13d-49b1-b536-1fb0840baeba None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:10.885314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f79e8-b13d-49b1-b536-1fb0840baeba None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:10.885790 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f79e8-b13d-49b1-b536-1fb0840baeba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:10.912681 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-608f79e8-b13d-49b1-b536-1fb0840baeba None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:10.915640 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-608f79e8-b13d-49b1-b536-1fb0840baeba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:10.951940 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:10.955705 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 240/458] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 14:07:10.980589 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9294fc-37ed-426c-bf44-309d7d4edb90 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:11.045058 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 219/459] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:07:11.084439 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608f79e8-b13d-49b1-b536-1fb0840baeba None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:11.117536 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9294fc-37ed-426c-bf44-309d7d4edb90 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:11.136814 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:11.155366 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85a5529-9a8a-4fe5-af0e-d65104ab7317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:11.156491 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a5529-9a8a-4fe5-af0e-d65104ab7317 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:11.156980 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a5529-9a8a-4fe5-af0e-d65104ab7317 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:11.157452 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a5529-9a8a-4fe5-af0e-d65104ab7317 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:11.169781 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85a5529-9a8a-4fe5-af0e-d65104ab7317 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:11.171691 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85a5529-9a8a-4fe5-af0e-d65104ab7317 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:11.194547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-a85a5529-9a8a-4fe5-af0e-d65104ab7317 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:11.194547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:11.194547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:11.194547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:11.194547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:11.194547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:11.194547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:11.194547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:11.194547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:11.194547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:11.194547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:11.194547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:11.194547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:11.194547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:11.194547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:11.195472 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:11.195472 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:11.195472 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:11.195472 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:11.195472 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:11.195472 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:11.195472 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:11.195472 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:11.195472 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:11.195472 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:11.195472 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:11.195472 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:11.195472 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:11.195472 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:11.195472 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:11.195472 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:11.195472 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:11.196319 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:11.196319 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:11.196319 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:11.196319 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:11.198718 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 241/460] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:07:11.200295 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1534e-3408-44aa-add9-d24743daae23 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:11.200864 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1534e-3408-44aa-add9-d24743daae23 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:11.201325 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1534e-3408-44aa-add9-d24743daae23 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:11.203039 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 242/461] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:11.204365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1534e-3408-44aa-add9-d24743daae23 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:11.212672 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608f79e8-b13d-49b1-b536-1fb0840baeba None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:11.257990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad60488-9562-4edd-ba16-54070f0f06ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:11.258636 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad60488-9562-4edd-ba16-54070f0f06ee None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:11.258944 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad60488-9562-4edd-ba16-54070f0f06ee None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:11.259227 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad60488-9562-4edd-ba16-54070f0f06ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:11.272938 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad60488-9562-4edd-ba16-54070f0f06ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:11.274805 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad60488-9562-4edd-ba16-54070f0f06ee None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:11.295585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-bad60488-9562-4edd-ba16-54070f0f06ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:11.295585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:11.295585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:11.295585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:11.295585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:11.295585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:11.295585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:11.295585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:11.295585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:11.295585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:11.295585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:11.295585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:11.295585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:11.295585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:11.295585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:11.296634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:11.296634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:11.296634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:11.296634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:11.296634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:11.296634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:11.296634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:11.296634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:11.296634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:11.296634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:11.296634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:11.296634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:11.296634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:11.296634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:11.296634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:11.296634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:11.296634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:11.297695 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:11.297695 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:11.297695 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:11.297695 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:11.298635 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 220/462] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:07:11.300158 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53f47e-4d63-4475-a8f2-bdd9cedeb8da None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:11.300542 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53f47e-4d63-4475-a8f2-bdd9cedeb8da None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:11.300882 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53f47e-4d63-4475-a8f2-bdd9cedeb8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:11.318896 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:11.324884 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-8a53f47e-4d63-4475-a8f2-bdd9cedeb8da None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:11.326726 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8a53f47e-4d63-4475-a8f2-bdd9cedeb8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:11.342193 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1534e-3408-44aa-add9-d24743daae23 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:11.388371 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba80bd25-f27a-42cf-bdaa-a96b7a47f953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:11.389275 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba80bd25-f27a-42cf-bdaa-a96b7a47f953 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:11.389556 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba80bd25-f27a-42cf-bdaa-a96b7a47f953 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:11.389929 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba80bd25-f27a-42cf-bdaa-a96b7a47f953 None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:11.390722 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba80bd25-f27a-42cf-bdaa-a96b7a47f953 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:11.391352 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba80bd25-f27a-42cf-bdaa-a96b7a47f953 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:11.392611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba80bd25-f27a-42cf-bdaa-a96b7a47f953 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:11.410759 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 243/463] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:11.412361 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba80bd25-f27a-42cf-bdaa-a96b7a47f953 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:11.425989 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 221/464] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:07:11.427184 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a53f47e-4d63-4475-a8f2-bdd9cedeb8da None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:11.443844 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:11.525789 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba80bd25-f27a-42cf-bdaa-a96b7a47f953 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:11.547491 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a53f47e-4d63-4475-a8f2-bdd9cedeb8da None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:11.563332 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a4c5e4-434f-4434-80d5-d03910e36b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:11.564058 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4c5e4-434f-4434-80d5-d03910e36b3e None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:11.564335 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4c5e4-434f-4434-80d5-d03910e36b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:11.564728 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4c5e4-434f-4434-80d5-d03910e36b3e None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:11.565416 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a4c5e4-434f-4434-80d5-d03910e36b3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:11.565769 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a4c5e4-434f-4434-80d5-d03910e36b3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:11.567232 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a4c5e4-434f-4434-80d5-d03910e36b3e None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:11.579641 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 244/465] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:11.580467 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a4c5e4-434f-4434-80d5-d03910e36b3e None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:11.586329 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e44834-79b6-4804-8583-5eb16e3700ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:11.586329 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e44834-79b6-4804-8583-5eb16e3700ac None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:11.586329 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e44834-79b6-4804-8583-5eb16e3700ac None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:11.586329 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e44834-79b6-4804-8583-5eb16e3700ac None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:11.586643 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e44834-79b6-4804-8583-5eb16e3700ac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:11.587643 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e44834-79b6-4804-8583-5eb16e3700ac None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:11.599759 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:11.669630 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:11.681561 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a4c5e4-434f-4434-80d5-d03910e36b3e None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:11.702451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 222/466] 10.222.0.26 () {60 vars in 1132 bytes} [Fri Aug 5 14:07:11 2022] POST /identity/v3/users => generated 248 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:07:11.703914 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e207c58-1aa5-47b6-83e9-2920ed0757b3 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:11.704248 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e207c58-1aa5-47b6-83e9-2920ed0757b3 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:11.704569 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e207c58-1aa5-47b6-83e9-2920ed0757b3 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:11.708486 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 223/467] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:11.708486 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e207c58-1aa5-47b6-83e9-2920ed0757b3 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:11.726077 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd90605-6fae-4172-a076-65e73d2a6325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:11.726812 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd90605-6fae-4172-a076-65e73d2a6325 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:11.727094 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd90605-6fae-4172-a076-65e73d2a6325 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:11.727362 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd90605-6fae-4172-a076-65e73d2a6325 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:11.739759 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd90605-6fae-4172-a076-65e73d2a6325 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:11.741336 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd90605-6fae-4172-a076-65e73d2a6325 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:11.763421 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-6fd90605-6fae-4172-a076-65e73d2a6325 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:11.763421 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:11.763421 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:11.763421 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:11.763421 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:11.763421 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:11.763421 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:11.763421 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:11.763421 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:11.763421 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:11.763421 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:11.763421 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:11.763421 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:11.763421 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:11.763421 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:11.768994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:11.768994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:11.768994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:11.768994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:11.768994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:11.768994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:11.768994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:11.768994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:11.768994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:11.768994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:11.768994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:11.768994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:11.768994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:11.768994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:11.768994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:11.768994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:11.768994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:11.769855 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:11.769855 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:11.769855 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:11.769855 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:11.769855 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 245/468] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:11.769855 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce731ed-0b43-4275-84bf-36fa8ffd535c None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:11.769855 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce731ed-0b43-4275-84bf-36fa8ffd535c None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:11.769855 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce731ed-0b43-4275-84bf-36fa8ffd535c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:11.797223 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-bce731ed-0b43-4275-84bf-36fa8ffd535c None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:11.798479 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-bce731ed-0b43-4275-84bf-36fa8ffd535c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:11.861443 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e207c58-1aa5-47b6-83e9-2920ed0757b3 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:11.878923 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 246/469] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:07:11.884432 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce731ed-0b43-4275-84bf-36fa8ffd535c None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:11.927167 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f369913-a6f3-4f4e-8283-72ce2be30057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:11.928017 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f369913-a6f3-4f4e-8283-72ce2be30057 None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:11.928491 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f369913-a6f3-4f4e-8283-72ce2be30057 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:11.928885 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f369913-a6f3-4f4e-8283-72ce2be30057 None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:11.929697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f369913-a6f3-4f4e-8283-72ce2be30057 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:11.931331 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f369913-a6f3-4f4e-8283-72ce2be30057 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:11.970627 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:12.002501 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce731ed-0b43-4275-84bf-36fa8ffd535c None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:12.008399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 224/470] 10.222.0.26 () {60 vars in 1132 bytes} [Fri Aug 5 14:07:11 2022] POST /identity/v3/users => generated 246 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:07:12.011290 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f369913-a6f3-4f4e-8283-72ce2be30057 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:12.057990 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b6105-2ef0-489b-ad3c-bfa4e208cc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:12.058636 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b6105-2ef0-489b-ad3c-bfa4e208cc87 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:12.058913 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b6105-2ef0-489b-ad3c-bfa4e208cc87 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:12.059233 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b6105-2ef0-489b-ad3c-bfa4e208cc87 None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:12.059940 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b6105-2ef0-489b-ad3c-bfa4e208cc87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:12.060244 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b6105-2ef0-489b-ad3c-bfa4e208cc87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:12.061734 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b6105-2ef0-489b-ad3c-bfa4e208cc87 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:12.071940 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:12.083953 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 247/471] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:12.091117 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b6105-2ef0-489b-ad3c-bfa4e208cc87 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:12.137243 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f369913-a6f3-4f4e-8283-72ce2be30057 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:12.182015 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a889a9-3379-4ea9-a70a-0115a7dd45d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:12.182809 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a889a9-3379-4ea9-a70a-0115a7dd45d2 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:12.183157 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a889a9-3379-4ea9-a70a-0115a7dd45d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:12.183501 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a889a9-3379-4ea9-a70a-0115a7dd45d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:12.199897 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a889a9-3379-4ea9-a70a-0115a7dd45d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:12.201692 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a889a9-3379-4ea9-a70a-0115a7dd45d2 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:12.222025 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-68a889a9-3379-4ea9-a70a-0115a7dd45d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:12.222025 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:12.222025 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:12.222025 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:12.222025 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:12.222025 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:12.222025 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:12.222025 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:12.222025 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:12.222025 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:12.222025 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:12.222025 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:12.222025 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:12.222025 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:12.222025 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:12.222955 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:12.222955 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:12.222955 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:12.222955 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:12.222955 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:12.222955 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:12.222955 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:12.222955 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:12.222955 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:12.222955 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:12.222955 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:12.222955 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:12.222955 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:12.222955 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:12.222955 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:12.222955 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:12.222955 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:12.224748 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:12.224748 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:12.224748 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:12.224748 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:12.228664 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 225/472] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:12.230370 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb01df-7e84-4599-aaff-760826d83188 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:12.230651 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb01df-7e84-4599-aaff-760826d83188 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:12.230924 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb01df-7e84-4599-aaff-760826d83188 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:12.232441 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 226/473] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:12.234635 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb01df-7e84-4599-aaff-760826d83188 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:12.251340 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b6105-2ef0-489b-ad3c-bfa4e208cc87 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:12.290223 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:12.298797 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b80f1-7b1f-499d-b294-3b492152fad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:12.299379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b80f1-7b1f-499d-b294-3b492152fad1 None admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:12.302862 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b80f1-7b1f-499d-b294-3b492152fad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:12.302862 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b80f1-7b1f-499d-b294-3b492152fad1 None admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:12.302862 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96b80f1-7b1f-499d-b294-3b492152fad1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:12.302862 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96b80f1-7b1f-499d-b294-3b492152fad1 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:12.367026 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:12.371010 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb01df-7e84-4599-aaff-760826d83188 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:12.392097 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 248/474] 10.222.0.26 () {60 vars in 1132 bytes} [Fri Aug 5 14:07:12 2022] POST /identity/v3/users => generated 248 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:07:12.394112 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d8c24-ebb1-4f60-ba14-ce481c28ecbf None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:12.394688 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d8c24-ebb1-4f60-ba14-ce481c28ecbf None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:12.395149 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d8c24-ebb1-4f60-ba14-ce481c28ecbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:12.419726 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-ae8d8c24-ebb1-4f60-ba14-ce481c28ecbf None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:12.421500 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717b8ce0-0235-4c6b-94b4-a3c67876ed22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:12.421732 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-ae8d8c24-ebb1-4f60-ba14-ce481c28ecbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:12.422130 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b8ce0-0235-4c6b-94b4-a3c67876ed22 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:12.422407 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b8ce0-0235-4c6b-94b4-a3c67876ed22 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:12.422681 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b8ce0-0235-4c6b-94b4-a3c67876ed22 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:12.423429 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717b8ce0-0235-4c6b-94b4-a3c67876ed22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:12.423734 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717b8ce0-0235-4c6b-94b4-a3c67876ed22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:12.425541 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717b8ce0-0235-4c6b-94b4-a3c67876ed22 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:12.472743 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 227/475] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:12.477240 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717b8ce0-0235-4c6b-94b4-a3c67876ed22 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:12.566128 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 249/476] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:07:12.596743 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d8c24-ebb1-4f60-ba14-ce481c28ecbf None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:12.658901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717b8ce0-0235-4c6b-94b4-a3c67876ed22 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:12.706779 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30740d-0b3d-4b8e-bd94-671ef23702e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:12.707398 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30740d-0b3d-4b8e-bd94-671ef23702e6 None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:12.708441 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30740d-0b3d-4b8e-bd94-671ef23702e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:12.708740 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30740d-0b3d-4b8e-bd94-671ef23702e6 None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:12.709746 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30740d-0b3d-4b8e-bd94-671ef23702e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:12.711355 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30740d-0b3d-4b8e-bd94-671ef23702e6 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:12.730615 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d8c24-ebb1-4f60-ba14-ce481c28ecbf None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:12.749886 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:12.769113 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b4396-1608-486c-9a6a-9a03a1241f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:12.769740 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b4396-1608-486c-9a6a-9a03a1241f39 None admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:12.770039 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b4396-1608-486c-9a6a-9a03a1241f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:12.770959 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b4396-1608-486c-9a6a-9a03a1241f39 None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:12.771658 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4b4396-1608-486c-9a6a-9a03a1241f39 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:12.773079 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4b4396-1608-486c-9a6a-9a03a1241f39 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:12.817049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.sql.core [None req-5e4b4396-1608-486c-9a6a-9a03a1241f39 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 05 14:07:12.817049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 05 14:07:12.817049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [parameters: {'id': 'c93f053287a142bb80cf898e51e4011d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 05 14:07:12.817049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80322) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 05 14:07:12.824399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 228/477] 10.222.0.26 () {60 vars in 1132 bytes} [Fri Aug 5 14:07:12 2022] POST /identity/v3/users => generated 249 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:07:12.835704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-5e4b4396-1608-486c-9a6a-9a03a1241f39 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 05 14:07:12.835704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:12.835704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 14:07:12.835704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 14:07:12.835704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 14:07:12.835704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 14:07:12.835704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 14:07:12.835704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 14:07:12.835704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 14:07:12.835704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application conn.query(q) Aug 05 14:07:12.835704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 14:07:12.835704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 14:07:12.835704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 14:07:12.835704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application result.read() Aug 05 14:07:12.835704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 14:07:12.837765 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 14:07:12.837765 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 14:07:12.837765 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 14:07:12.837765 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 14:07:12.837765 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 14:07:12.837765 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 14:07:12.837765 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 14:07:12.837765 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 05 14:07:12.837765 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:12.837765 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 05 14:07:12.837765 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:12.837765 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:12.837765 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 05 14:07:12.837765 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 05 14:07:12.837765 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 05 14:07:12.837765 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return ref.to_dict() Aug 05 14:07:12.837765 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 14:07:12.837765 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application next(self.gen) Aug 05 14:07:12.839706 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 05 14:07:12.839706 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application yield resource Aug 05 14:07:12.839706 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 14:07:12.839706 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application next(self.gen) Aug 05 14:07:12.839706 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 05 14:07:12.839706 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self.session.rollback() Aug 05 14:07:12.839706 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 05 14:07:12.839706 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self.force_reraise() Aug 05 14:07:12.839706 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 05 14:07:12.839706 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise self.value Aug 05 14:07:12.839706 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 05 14:07:12.839706 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 05 14:07:12.839706 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 05 14:07:12.839706 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application session.commit() Aug 05 14:07:12.839706 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 05 14:07:12.839706 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 05 14:07:12.839706 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 05 14:07:12.841942 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self._prepare_impl() Aug 05 14:07:12.841942 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 05 14:07:12.841942 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self.session.flush() Aug 05 14:07:12.841942 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 05 14:07:12.841942 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self._flush(objects) Aug 05 14:07:12.841942 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 05 14:07:12.841942 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 05 14:07:12.841942 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 05 14:07:12.841942 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application compat.raise_( Aug 05 14:07:12.841942 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 14:07:12.841942 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception Aug 05 14:07:12.841942 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 05 14:07:12.841942 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application flush_context.execute() Aug 05 14:07:12.841942 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 05 14:07:12.841942 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rec.execute(self) Aug 05 14:07:12.841942 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 05 14:07:12.841942 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 05 14:07:12.844036 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 05 14:07:12.844036 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 05 14:07:12.844036 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 05 14:07:12.844036 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 05 14:07:12.844036 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 05 14:07:12.844036 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 05 14:07:12.844036 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 05 14:07:12.844036 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 05 14:07:12.844036 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 05 14:07:12.844036 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 05 14:07:12.844036 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 05 14:07:12.844036 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 05 14:07:12.844036 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 05 14:07:12.844036 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 05 14:07:12.844036 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 14:07:12.844036 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception Aug 05 14:07:12.846097 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 14:07:12.846097 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 14:07:12.846097 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 14:07:12.846097 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 14:07:12.846097 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 14:07:12.846097 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 14:07:12.846097 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 14:07:12.846097 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application conn.query(q) Aug 05 14:07:12.846097 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 14:07:12.846097 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 14:07:12.846097 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 14:07:12.846097 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application result.read() Aug 05 14:07:12.846097 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 14:07:12.846097 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 14:07:12.846097 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 14:07:12.846097 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 14:07:12.854213 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 14:07:12.854213 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 14:07:12.854213 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 14:07:12.854213 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 14:07:12.854213 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 05 14:07:12.854213 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 05 14:07:12.854213 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [parameters: {'id': 'c93f053287a142bb80cf898e51e4011d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 05 14:07:12.854213 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 05 14:07:12.854213 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:12.854213 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 14:07:12.854213 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:12.854213 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:12.854213 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:12.854213 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:12.854213 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:12.854213 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:12.856133 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:12.856133 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:12.856133 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:12.856133 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:12.856133 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:12.856133 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:12.856133 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 05 14:07:12.856133 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 05 14:07:12.856133 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:12.856133 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:12.856133 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 05 14:07:12.856133 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 05 14:07:12.856133 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 05 14:07:12.856133 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 05 14:07:12.856133 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 05 14:07:12.856133 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:12.858147 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 250/478] 10.222.0.26 () {60 vars in 1131 bytes} [Fri Aug 5 14:07:12 2022] POST /identity/v3/roles => generated 144 bytes in 241 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:12.858282 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30740d-0b3d-4b8e-bd94-671ef23702e6 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:12.877778 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:13.010923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30740d-0b3d-4b8e-bd94-671ef23702e6 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:13.053360 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f537d9f-df66-4b45-9ce7-95c77dfdaea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:13.054007 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f537d9f-df66-4b45-9ce7-95c77dfdaea7 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:13.054281 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f537d9f-df66-4b45-9ce7-95c77dfdaea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:13.054552 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f537d9f-df66-4b45-9ce7-95c77dfdaea7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:13.065133 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f537d9f-df66-4b45-9ce7-95c77dfdaea7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:07:13.070716 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-0f537d9f-df66-4b45-9ce7-95c77dfdaea7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 14:07:13.070716 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:13.070716 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:13.070716 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:13.070716 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:13.070716 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:13.070716 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:13.070716 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:13.070716 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:13.070716 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:13.070716 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:13.070716 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:13.070716 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:07:13.070716 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:07:13.070716 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:07:13.070716 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:07:13.072824 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:07:13.072824 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:07:13.072824 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:07:13.072824 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:07:13.072824 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:13.072824 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:13.072824 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:13.072824 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:13.072824 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:13.072824 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:13.072824 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:13.072824 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:13.072824 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:13.072824 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:13.072824 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:13.072824 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:13.074604 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:13.074604 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:13.074604 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:13.074604 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:13.074604 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:07:13.074604 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:07:13.074604 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:07:13.074604 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:07:13.074604 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:07:13.074604 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:07:13.074604 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 14:07:13.074604 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:13.074604 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 229/479] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Aug 5 14:07:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:13.077610 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b4396-1608-486c-9a6a-9a03a1241f39 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:13.211255 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b4396-1608-486c-9a6a-9a03a1241f39 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:13.258188 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0ca18-1c6a-4356-9b60-c2249a9fcf6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:13.258814 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0ca18-1c6a-4356-9b60-c2249a9fcf6e None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:13.259092 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0ca18-1c6a-4356-9b60-c2249a9fcf6e None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:13.259370 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0ca18-1c6a-4356-9b60-c2249a9fcf6e None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:13.260088 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a0ca18-1c6a-4356-9b60-c2249a9fcf6e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:13.260395 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a0ca18-1c6a-4356-9b60-c2249a9fcf6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:13.261860 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a0ca18-1c6a-4356-9b60-c2249a9fcf6e None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:13.273180 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 251/480] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:07:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:13.956229 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2683c92-7054-418d-a5f6-902912bf6bfc None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:13.956787 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2683c92-7054-418d-a5f6-902912bf6bfc None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:13.956787 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2683c92-7054-418d-a5f6-902912bf6bfc None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:13.958888 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 230/481] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:13.964515 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be5bea-8682-4503-9912-fcdf88e8e76a None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:13.964804 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be5bea-8682-4503-9912-fcdf88e8e76a None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:13.965083 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be5bea-8682-4503-9912-fcdf88e8e76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:13.985459 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-95be5bea-8682-4503-9912-fcdf88e8e76a None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:13.986649 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-95be5bea-8682-4503-9912-fcdf88e8e76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:14.037514 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:14.074655 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 252/482] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:07:14.098414 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2683c92-7054-418d-a5f6-902912bf6bfc None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:14.195415 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:14.221878 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2683c92-7054-418d-a5f6-902912bf6bfc None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:14.271183 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ec3e4-5117-4d1d-ae05-ae1f5fdf44cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:14.271888 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ec3e4-5117-4d1d-ae05-ae1f5fdf44cf None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:14.272169 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ec3e4-5117-4d1d-ae05-ae1f5fdf44cf None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:14.272465 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ec3e4-5117-4d1d-ae05-ae1f5fdf44cf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:14.287458 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835ec3e4-5117-4d1d-ae05-ae1f5fdf44cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:07:14.292782 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-835ec3e4-5117-4d1d-ae05-ae1f5fdf44cf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:07:14.292782 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:14.292782 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:14.292782 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:14.292782 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:14.292782 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:14.292782 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:14.292782 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:14.292782 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:14.292782 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:14.292782 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:14.292782 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:14.292782 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:07:14.292782 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:07:14.292782 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:07:14.292782 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:07:14.293645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:07:14.293645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:07:14.293645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:07:14.293645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:07:14.293645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:14.293645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:14.293645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:14.293645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:14.293645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:14.293645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:14.293645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:14.293645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:14.293645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:14.293645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:14.293645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:14.293645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:14.294456 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:14.294456 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:14.294456 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:14.294456 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:14.294456 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:07:14.294456 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:07:14.294456 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:07:14.294456 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:07:14.294456 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:07:14.294456 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:07:14.294456 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:07:14.294456 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:14.294456 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 231/483] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Aug 5 14:07:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:14.300089 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be5bea-8682-4503-9912-fcdf88e8e76a None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:14.425540 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be5bea-8682-4503-9912-fcdf88e8e76a None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:14.472744 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c6bcd-e20c-4b78-af58-6053082887c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:14.474001 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c6bcd-e20c-4b78-af58-6053082887c4 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:14.474283 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c6bcd-e20c-4b78-af58-6053082887c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:14.474597 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c6bcd-e20c-4b78-af58-6053082887c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:14.475316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13c6bcd-e20c-4b78-af58-6053082887c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:14.475614 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13c6bcd-e20c-4b78-af58-6053082887c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:14.477010 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13c6bcd-e20c-4b78-af58-6053082887c4 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:14.491204 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 253/484] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Aug 5 14:07:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:14.496937 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ec3e4-5117-4d1d-ae05-ae1f5fdf44cf None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:14.525125 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452cf0d-1551-4a64-bf2b-cffc604a5941 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:14.525125 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452cf0d-1551-4a64-bf2b-cffc604a5941 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:14.525125 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452cf0d-1551-4a64-bf2b-cffc604a5941 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:14.525999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 254/485] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:14.533153 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bfe53b-7898-4159-8f42-3f9ab2c027a6 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:14.533661 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bfe53b-7898-4159-8f42-3f9ab2c027a6 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:14.534079 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bfe53b-7898-4159-8f42-3f9ab2c027a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:14.565189 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-b8bfe53b-7898-4159-8f42-3f9ab2c027a6 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:14.567532 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-b8bfe53b-7898-4159-8f42-3f9ab2c027a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:14.618452 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:14.657552 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ec3e4-5117-4d1d-ae05-ae1f5fdf44cf None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:14.673935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 255/486] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 14:07:14.703936 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de4fa3-3b77-4696-bacf-897e76507899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:14.704766 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de4fa3-3b77-4696-bacf-897e76507899 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:14.705177 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de4fa3-3b77-4696-bacf-897e76507899 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:14.705563 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de4fa3-3b77-4696-bacf-897e76507899 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:14.710491 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bfe53b-7898-4159-8f42-3f9ab2c027a6 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:14.724900 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:14.735828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5de4fa3-3b77-4696-bacf-897e76507899 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:14.738286 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5de4fa3-3b77-4696-bacf-897e76507899 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:14.772545 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-c5de4fa3-3b77-4696-bacf-897e76507899 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:14.772545 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:14.772545 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:14.772545 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:14.772545 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:14.772545 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:14.772545 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:14.772545 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:14.772545 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:14.772545 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:14.772545 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:14.772545 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:14.772545 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:14.772545 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:14.772545 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:14.773453 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:14.773453 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:14.773453 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:14.773453 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:14.773453 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:14.773453 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:14.773453 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:14.773453 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:14.773453 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:14.773453 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:14.773453 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:14.773453 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:14.773453 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:14.773453 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:14.773453 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:14.773453 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:14.773453 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:14.774399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:14.774399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:14.774399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:14.774399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:14.777470 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 232/487] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:14.792472 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de4fa3-3b77-4696-bacf-897e76507899 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:14.849272 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bfe53b-7898-4159-8f42-3f9ab2c027a6 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:14.886084 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a832d92-ed15-485b-8232-289c26d86377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:14.886830 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a832d92-ed15-485b-8232-289c26d86377 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:14.887110 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a832d92-ed15-485b-8232-289c26d86377 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:14.887428 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a832d92-ed15-485b-8232-289c26d86377 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:14.898623 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a832d92-ed15-485b-8232-289c26d86377 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:07:14.903652 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-4a832d92-ed15-485b-8232-289c26d86377 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:07:14.903652 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:14.903652 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:14.903652 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:14.903652 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:14.903652 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:14.903652 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:14.903652 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:14.903652 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:14.903652 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:14.903652 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:14.903652 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:14.903652 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:07:14.903652 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:07:14.903652 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:07:14.903652 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:07:14.905698 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:07:14.905698 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:07:14.905698 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:07:14.905698 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:07:14.905698 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:14.905698 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:14.905698 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:14.905698 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:14.905698 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:14.905698 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:14.905698 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:14.905698 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:14.905698 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:14.905698 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:14.905698 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:14.905698 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:14.907582 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:14.907582 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:14.907582 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:14.907582 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:14.907582 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:07:14.907582 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:07:14.907582 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:07:14.907582 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:07:14.907582 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:07:14.907582 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:07:14.907582 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:07:14.907582 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:14.907582 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 256/488] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Aug 5 14:07:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:07:14.915347 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a832d92-ed15-485b-8232-289c26d86377 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:14.971151 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:14.984702 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de4fa3-3b77-4696-bacf-897e76507899 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:15.024457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d350d-9a95-4db9-b5e1-7f02c90e6b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:15.025444 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d350d-9a95-4db9-b5e1-7f02c90e6b1c None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:15.025921 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d350d-9a95-4db9-b5e1-7f02c90e6b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:15.026416 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d350d-9a95-4db9-b5e1-7f02c90e6b1c None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:15.027355 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d350d-9a95-4db9-b5e1-7f02c90e6b1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:15.027921 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d350d-9a95-4db9-b5e1-7f02c90e6b1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:15.029580 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d350d-9a95-4db9-b5e1-7f02c90e6b1c None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:15.047021 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 233/489] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:15.053491 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d350d-9a95-4db9-b5e1-7f02c90e6b1c None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:15.108846 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:15.164598 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d350d-9a95-4db9-b5e1-7f02c90e6b1c None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:15.193109 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a832d92-ed15-485b-8232-289c26d86377 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:15.201062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b7e2f-9400-4e29-b821-02a4f8ae9882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:15.201969 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b7e2f-9400-4e29-b821-02a4f8ae9882 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:15.202472 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b7e2f-9400-4e29-b821-02a4f8ae9882 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:15.203007 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b7e2f-9400-4e29-b821-02a4f8ae9882 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:15.211769 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3b7e2f-9400-4e29-b821-02a4f8ae9882 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:15.213705 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3b7e2f-9400-4e29-b821-02a4f8ae9882 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:15.228444 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-ae3b7e2f-9400-4e29-b821-02a4f8ae9882 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 05 14:07:15.228444 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:15.228444 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:15.228444 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:15.228444 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:15.228444 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:15.228444 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:15.228444 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:15.228444 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:15.228444 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:15.228444 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:15.228444 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:15.228444 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 14:07:15.228444 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 14:07:15.228444 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 14:07:15.228444 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 14:07:15.229394 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:15.229394 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:15.229394 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:07:15.229394 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:15.229394 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:07:15.229394 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:15.229394 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:15.229394 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:15.229394 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:15.229394 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:15.229394 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:15.229394 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:15.229394 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:15.229394 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:15.229394 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:15.229394 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:15.229394 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:15.229394 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:15.230305 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:15.230305 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:15.230305 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 14:07:15.230305 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 14:07:15.230305 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 14:07:15.230305 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 14:07:15.230305 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 14:07:15.230305 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 14:07:15.230305 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 05 14:07:15.230305 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:15.233115 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 234/490] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:07:15 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:15.239123 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b7e2f-9400-4e29-b821-02a4f8ae9882 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:15.276339 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09666380-2925-4b64-8152-bc736c2f2056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:15.277181 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09666380-2925-4b64-8152-bc736c2f2056 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:15.277464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09666380-2925-4b64-8152-bc736c2f2056 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:15.277738 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09666380-2925-4b64-8152-bc736c2f2056 None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:15.279344 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09666380-2925-4b64-8152-bc736c2f2056 None admin] RBAC: Adding query filter params (name=service) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:15.279656 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09666380-2925-4b64-8152-bc736c2f2056 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:15.281061 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09666380-2925-4b64-8152-bc736c2f2056 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:15.291131 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 257/491] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Aug 5 14:07:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:15.292836 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce84d3-e1ea-4ce9-a1dc-7f7dcd0a5ca0 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:15.293345 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce84d3-e1ea-4ce9-a1dc-7f7dcd0a5ca0 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:15.293816 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce84d3-e1ea-4ce9-a1dc-7f7dcd0a5ca0 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:15.295199 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 258/492] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:15.297894 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce84d3-e1ea-4ce9-a1dc-7f7dcd0a5ca0 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:15.358085 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b7e2f-9400-4e29-b821-02a4f8ae9882 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:15.367377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:15.392801 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edbc415-5170-4cda-ab47-60002722244e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:15.393421 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbc415-5170-4cda-ab47-60002722244e None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:15.393686 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbc415-5170-4cda-ab47-60002722244e None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:15.394015 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbc415-5170-4cda-ab47-60002722244e None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:15.394705 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:15.395783 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edbc415-5170-4cda-ab47-60002722244e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:15.395783 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edbc415-5170-4cda-ab47-60002722244e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:15.397303 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edbc415-5170-4cda-ab47-60002722244e None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:15.411378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce84d3-e1ea-4ce9-a1dc-7f7dcd0a5ca0 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:15.435649 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 235/493] 10.222.0.26 () {56 vars in 1137 bytes} [Fri Aug 5 14:07:15 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:15.437077 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131f93f-e441-4b5d-abe0-7a8754fdfc05 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:15.437359 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131f93f-e441-4b5d-abe0-7a8754fdfc05 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:15.437632 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131f93f-e441-4b5d-abe0-7a8754fdfc05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:15.448563 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983baad1-0d87-41e8-8b69-9c46ce3515ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:15.449205 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983baad1-0d87-41e8-8b69-9c46ce3515ee None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:15.449484 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983baad1-0d87-41e8-8b69-9c46ce3515ee None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:15.449752 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983baad1-0d87-41e8-8b69-9c46ce3515ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:15.456691 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-8131f93f-e441-4b5d-abe0-7a8754fdfc05 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:15.458360 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8131f93f-e441-4b5d-abe0-7a8754fdfc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:15.460990 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983baad1-0d87-41e8-8b69-9c46ce3515ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:15.462514 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983baad1-0d87-41e8-8b69-9c46ce3515ee None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:15.478767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-983baad1-0d87-41e8-8b69-9c46ce3515ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:15.478767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:15.478767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:15.478767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:15.478767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:15.478767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:15.478767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:15.478767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:15.478767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:15.478767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:15.478767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:15.478767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:15.478767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:15.478767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:15.478767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:15.479658 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:15.479658 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:15.479658 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:15.479658 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:15.479658 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:15.479658 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:15.479658 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:15.479658 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:15.479658 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:15.479658 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:15.479658 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:15.479658 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:15.479658 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:15.479658 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:15.479658 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:15.479658 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:15.479658 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:15.481396 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:15.481396 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:15.481396 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:15.481396 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:15.481396 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 259/494] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:15.481833 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff4e5b-29ab-4f37-b3cf-5b75e3d2de35 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:15.482047 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff4e5b-29ab-4f37-b3cf-5b75e3d2de35 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:15.482317 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff4e5b-29ab-4f37-b3cf-5b75e3d2de35 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:15.483683 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 260/495] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:15.488076 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff4e5b-29ab-4f37-b3cf-5b75e3d2de35 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:15.535706 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 236/496] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:07:15.536862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131f93f-e441-4b5d-abe0-7a8754fdfc05 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:15.614896 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff4e5b-29ab-4f37-b3cf-5b75e3d2de35 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:15.638176 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:15.644336 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131f93f-e441-4b5d-abe0-7a8754fdfc05 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:15.651849 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798ec355-18a9-4155-a5f9-3a19d8fb7e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:15.652637 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ec355-18a9-4155-a5f9-3a19d8fb7e1a None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:15.652920 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ec355-18a9-4155-a5f9-3a19d8fb7e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:15.653190 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ec355-18a9-4155-a5f9-3a19d8fb7e1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:15.662382 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:15.681379 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0988ad3-8fc1-42b7-87ad-020fb4f8c9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:15.681997 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0988ad3-8fc1-42b7-87ad-020fb4f8c9ec None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:15.682279 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0988ad3-8fc1-42b7-87ad-020fb4f8c9ec None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:15.682551 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0988ad3-8fc1-42b7-87ad-020fb4f8c9ec None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:15.683244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0988ad3-8fc1-42b7-87ad-020fb4f8c9ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:15.683549 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0988ad3-8fc1-42b7-87ad-020fb4f8c9ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:15.684963 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0988ad3-8fc1-42b7-87ad-020fb4f8c9ec None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:15.701360 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 237/497] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:15.701360 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be350b76-1d13-4c1e-a623-72e2b5726305 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:15.701360 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be350b76-1d13-4c1e-a623-72e2b5726305 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:15.701360 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be350b76-1d13-4c1e-a623-72e2b5726305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:15.717284 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-be350b76-1d13-4c1e-a623-72e2b5726305 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:15.718480 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-be350b76-1d13-4c1e-a623-72e2b5726305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:15.730680 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798ec355-18a9-4155-a5f9-3a19d8fb7e1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:15.732495 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798ec355-18a9-4155-a5f9-3a19d8fb7e1a None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:15.792658 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 238/498] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:07:15.792658 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be350b76-1d13-4c1e-a623-72e2b5726305 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:15.841738 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 261/499] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:07:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 14:07:15.842796 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798ec355-18a9-4155-a5f9-3a19d8fb7e1a None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:15.884870 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:15.886055 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:15.900598 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be350b76-1d13-4c1e-a623-72e2b5726305 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:15.937127 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfa8bee-b585-4714-b572-3f0afe3ae0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:15.938037 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa8bee-b585-4714-b572-3f0afe3ae0e2 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:15.938489 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa8bee-b585-4714-b572-3f0afe3ae0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:15.938906 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa8bee-b585-4714-b572-3f0afe3ae0e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:15.948429 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798ec355-18a9-4155-a5f9-3a19d8fb7e1a None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:15.949951 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfa8bee-b585-4714-b572-3f0afe3ae0e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:07:15.955227 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-3dfa8bee-b585-4714-b572-3f0afe3ae0e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:07:15.955227 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:15.955227 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:15.955227 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:15.955227 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:15.955227 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:15.955227 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:15.955227 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:15.955227 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:15.955227 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:15.955227 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:15.955227 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:15.955227 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:07:15.955227 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:07:15.955227 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:07:15.955227 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:07:15.956326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:07:15.956326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:07:15.956326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:07:15.956326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:07:15.956326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:15.956326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:15.956326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:15.956326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:15.956326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:15.956326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:15.956326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:15.956326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:15.956326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:15.956326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:15.956326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:15.956326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:15.957344 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:15.957344 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:15.957344 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:15.957344 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:15.957344 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:07:15.957344 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:07:15.957344 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:07:15.957344 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:07:15.957344 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:07:15.957344 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:07:15.957344 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:07:15.957344 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:15.965491 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 239/500] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Aug 5 14:07:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:15.965491 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfa8bee-b585-4714-b572-3f0afe3ae0e2 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:15.986814 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8939eba-7c65-45d1-866c-2949df031c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:15.987459 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8939eba-7c65-45d1-866c-2949df031c26 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:15.987895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8939eba-7c65-45d1-866c-2949df031c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:15.988169 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8939eba-7c65-45d1-866c-2949df031c26 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:15.998859 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8939eba-7c65-45d1-866c-2949df031c26 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:16.001189 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8939eba-7c65-45d1-866c-2949df031c26 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:16.015065 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-b8939eba-7c65-45d1-866c-2949df031c26 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 14:07:16.015065 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:16.015065 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:16.015065 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:16.015065 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:16.015065 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:16.015065 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:16.015065 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:16.015065 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:16.015065 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:16.015065 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:16.015065 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:16.015065 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 14:07:16.015065 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 14:07:16.015065 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 14:07:16.015065 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 14:07:16.015997 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:16.015997 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:16.015997 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:07:16.015997 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:16.015997 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:07:16.015997 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:16.015997 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:16.015997 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:16.015997 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:16.015997 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:16.015997 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:16.015997 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:16.015997 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:16.015997 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:16.015997 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:16.015997 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:16.015997 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:16.015997 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:16.017459 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:16.017459 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:16.017459 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 14:07:16.017459 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 14:07:16.017459 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 14:07:16.017459 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 14:07:16.017459 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 14:07:16.017459 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 14:07:16.017459 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 14:07:16.017459 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:16.017459 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 262/501] 10.222.0.26 () {56 vars in 1093 bytes} [Fri Aug 5 14:07:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:16.017459 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8939eba-7c65-45d1-866c-2949df031c26 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:16.064103 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfa8bee-b585-4714-b572-3f0afe3ae0e2 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:16.097699 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5918a4-54c1-4475-a8df-f74ba51595d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:16.098963 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5918a4-54c1-4475-a8df-f74ba51595d3 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:16.099316 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5918a4-54c1-4475-a8df-f74ba51595d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:16.099505 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5918a4-54c1-4475-a8df-f74ba51595d3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:16.109289 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5918a4-54c1-4475-a8df-f74ba51595d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:07:16.112804 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:16.114692 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-1d5918a4-54c1-4475-a8df-f74ba51595d3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:07:16.114692 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:16.114692 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:16.114692 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:16.114692 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:16.114692 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:16.114692 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:16.114692 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:16.114692 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:16.114692 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:16.114692 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:16.114692 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:16.114692 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:07:16.114692 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:07:16.114692 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:07:16.114692 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:07:16.115546 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:07:16.115546 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:07:16.115546 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:07:16.115546 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:07:16.115546 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:16.115546 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:16.115546 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:16.115546 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:16.115546 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:16.115546 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:16.115546 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:16.115546 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:16.115546 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:16.115546 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:16.115546 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:16.115546 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:16.116356 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:16.116356 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:16.116356 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:16.116356 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:16.116356 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:07:16.116356 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:07:16.116356 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:07:16.116356 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:07:16.116356 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:07:16.116356 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:07:16.116356 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:07:16.116356 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:16.116356 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 240/502] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Aug 5 14:07:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:16.117051 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5918a4-54c1-4475-a8df-f74ba51595d3 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:16.117322 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8939eba-7c65-45d1-866c-2949df031c26 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:16.129661 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:16.150345 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e9af6e-f543-40bd-a263-b21d4960d883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:16.151165 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e9af6e-f543-40bd-a263-b21d4960d883 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:16.151614 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e9af6e-f543-40bd-a263-b21d4960d883 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:16.152239 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e9af6e-f543-40bd-a263-b21d4960d883 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:16.163457 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e9af6e-f543-40bd-a263-b21d4960d883 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:16.164939 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e9af6e-f543-40bd-a263-b21d4960d883 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:16.179943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-01e9af6e-f543-40bd-a263-b21d4960d883 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:16.179943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:16.179943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:16.179943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:16.179943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:16.179943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:16.179943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:16.179943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:16.179943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:16.179943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:16.179943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:16.179943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:16.179943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:16.179943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:16.179943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:16.180829 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:16.180829 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:16.180829 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:16.180829 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:16.180829 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:16.180829 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:16.180829 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:16.180829 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:16.180829 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:16.180829 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:16.180829 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:16.180829 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:16.180829 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:16.180829 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:16.180829 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:16.180829 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:16.180829 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:16.181692 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:16.181692 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:16.181692 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:16.181692 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:16.181692 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 263/503] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:16.182404 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963d159-a264-4022-9800-e9356c651704 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:16.182680 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963d159-a264-4022-9800-e9356c651704 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:16.182952 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963d159-a264-4022-9800-e9356c651704 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:16.184165 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 264/504] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:16.184874 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6963d159-a264-4022-9800-e9356c651704 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:16.223792 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5918a4-54c1-4475-a8df-f74ba51595d3 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:16.258023 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c4f721-8bfe-475d-b265-2754d3bc5472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:16.258627 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4f721-8bfe-475d-b265-2754d3bc5472 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:16.258899 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4f721-8bfe-475d-b265-2754d3bc5472 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:16.259165 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4f721-8bfe-475d-b265-2754d3bc5472 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:16.259860 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c4f721-8bfe-475d-b265-2754d3bc5472 None admin] RBAC: Adding query filter params (name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:16.260154 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c4f721-8bfe-475d-b265-2754d3bc5472 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:16.261542 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c4f721-8bfe-475d-b265-2754d3bc5472 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:16.269647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 241/505] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Aug 5 14:07:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:16.270333 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c4f721-8bfe-475d-b265-2754d3bc5472 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:16.286679 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6963d159-a264-4022-9800-e9356c651704 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:16.319115 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b621e-5287-441f-b363-10746be5fabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:16.319645 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b621e-5287-441f-b363-10746be5fabb None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:16.319916 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b621e-5287-441f-b363-10746be5fabb None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:16.320185 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b621e-5287-441f-b363-10746be5fabb None admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:16.320988 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b621e-5287-441f-b363-10746be5fabb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:16.321290 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b621e-5287-441f-b363-10746be5fabb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:16.322701 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b621e-5287-441f-b363-10746be5fabb None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:16.335804 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:16.358914 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 265/506] 10.222.0.26 () {56 vars in 1133 bytes} [Fri Aug 5 14:07:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:16.359742 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b621e-5287-441f-b363-10746be5fabb None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:16.370695 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:16.373983 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c4f721-8bfe-475d-b265-2754d3bc5472 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:16.407382 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e1fea-b446-4a5c-b34b-1081c850dd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:16.407991 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e1fea-b446-4a5c-b34b-1081c850dd1d None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:16.408271 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e1fea-b446-4a5c-b34b-1081c850dd1d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:16.408574 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e1fea-b446-4a5c-b34b-1081c850dd1d None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:16.409279 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2e1fea-b446-4a5c-b34b-1081c850dd1d None admin] RBAC: Adding query filter params (name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:16.409578 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2e1fea-b446-4a5c-b34b-1081c850dd1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:16.411520 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2e1fea-b446-4a5c-b34b-1081c850dd1d None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:16.419452 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 242/507] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Aug 5 14:07:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:16.420643 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d5f36-5474-4e4d-9308-2b21b083bf1d None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:16.420925 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d5f36-5474-4e4d-9308-2b21b083bf1d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:16.421196 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d5f36-5474-4e4d-9308-2b21b083bf1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:16.438923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-5a4d5f36-5474-4e4d-9308-2b21b083bf1d None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:16.440123 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5a4d5f36-5474-4e4d-9308-2b21b083bf1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:16.460325 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b621e-5287-441f-b363-10746be5fabb None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:16.494361 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13badf32-4fb0-4208-9154-de4b26334a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:16.494888 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13badf32-4fb0-4208-9154-de4b26334a6d None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:16.495166 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13badf32-4fb0-4208-9154-de4b26334a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:16.495439 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13badf32-4fb0-4208-9154-de4b26334a6d None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:16.496113 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13badf32-4fb0-4208-9154-de4b26334a6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:16.496428 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13badf32-4fb0-4208-9154-de4b26334a6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:16.498045 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13badf32-4fb0-4208-9154-de4b26334a6d None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:16.505425 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 243/508] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:07:16.506251 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d5f36-5474-4e4d-9308-2b21b083bf1d None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:16.518930 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 266/509] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:16.519822 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13badf32-4fb0-4208-9154-de4b26334a6d None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:16.573732 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:16.586555 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:16.610468 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d5f36-5474-4e4d-9308-2b21b083bf1d None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:16.619468 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13badf32-4fb0-4208-9154-de4b26334a6d None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:16.645670 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff77dd1-b0d8-4aa9-a891-997115b23cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:16.646361 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff77dd1-b0d8-4aa9-a891-997115b23cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:16.646642 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff77dd1-b0d8-4aa9-a891-997115b23cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:16.646915 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff77dd1-b0d8-4aa9-a891-997115b23cf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:16.652027 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597ce779-6b2a-4cd4-b5fd-a882495739ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:16.652709 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ce779-6b2a-4cd4-b5fd-a882495739ae None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:16.652984 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ce779-6b2a-4cd4-b5fd-a882495739ae None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:16.653256 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ce779-6b2a-4cd4-b5fd-a882495739ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:16.656765 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff77dd1-b0d8-4aa9-a891-997115b23cf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:16.658214 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff77dd1-b0d8-4aa9-a891-997115b23cf9 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:16.672452 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-2ff77dd1-b0d8-4aa9-a891-997115b23cf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:16.672452 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:16.672452 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:16.672452 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:16.672452 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:16.672452 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:16.672452 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:16.672452 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:16.672452 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:16.672452 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:16.672452 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:16.672452 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:16.672452 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:16.672452 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:16.672452 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:16.673324 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:16.673324 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:16.673324 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:16.673324 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:16.673324 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:16.673324 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:16.673324 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:16.673324 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:16.673324 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:16.673324 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:16.673324 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:16.673324 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:16.673324 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:16.673324 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:16.673324 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:16.673324 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:16.673324 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:16.674210 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:16.674210 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:16.674210 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:16.674210 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:16.674210 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 244/510] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:16.674766 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff77dd1-b0d8-4aa9-a891-997115b23cf9 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:16.724945 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597ce779-6b2a-4cd4-b5fd-a882495739ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:16.726422 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597ce779-6b2a-4cd4-b5fd-a882495739ae None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:16.776557 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff77dd1-b0d8-4aa9-a891-997115b23cf9 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:16.788893 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:16.790023 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:16.809786 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef17b4-2759-4eba-b53d-ec50427dd97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:16.810418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef17b4-2759-4eba-b53d-ec50427dd97a None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:16.810693 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef17b4-2759-4eba-b53d-ec50427dd97a None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:16.810964 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef17b4-2759-4eba-b53d-ec50427dd97a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:16.822088 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef17b4-2759-4eba-b53d-ec50427dd97a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:16.823599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef17b4-2759-4eba-b53d-ec50427dd97a None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:16.829353 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 267/511] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:07:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 14:07:16.829353 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597ce779-6b2a-4cd4-b5fd-a882495739ae None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:16.841843 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-acef17b4-2759-4eba-b53d-ec50427dd97a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:16.841843 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:16.841843 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:16.841843 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:16.841843 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:16.841843 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:16.841843 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:16.841843 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:16.841843 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:16.841843 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:16.841843 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:16.841843 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:16.841843 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:16.841843 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:16.841843 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:16.842864 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:16.842864 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:16.842864 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:16.842864 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:16.842864 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:16.842864 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:16.842864 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:16.842864 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:16.842864 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:16.842864 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:16.842864 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:16.842864 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:16.842864 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:16.842864 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:16.842864 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:16.842864 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:16.842864 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:16.843759 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:16.843759 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:16.843759 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:16.843759 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:16.843759 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 245/512] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:16.844018 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef17b4-2759-4eba-b53d-ec50427dd97a None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:16.922404 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597ce779-6b2a-4cd4-b5fd-a882495739ae None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:16.942816 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef17b4-2759-4eba-b53d-ec50427dd97a None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:16.956923 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e98b9b-850b-487f-bc6e-c4dbce42a129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:16.957597 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e98b9b-850b-487f-bc6e-c4dbce42a129 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:16.958093 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e98b9b-850b-487f-bc6e-c4dbce42a129 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:16.958361 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e98b9b-850b-487f-bc6e-c4dbce42a129 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:16.964741 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e98b9b-850b-487f-bc6e-c4dbce42a129 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:16.966155 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e98b9b-850b-487f-bc6e-c4dbce42a129 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:16.975568 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe9cf2-cf38-4e7d-8f65-57c93e23b860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:16.976188 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe9cf2-cf38-4e7d-8f65-57c93e23b860 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:16.976491 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe9cf2-cf38-4e7d-8f65-57c93e23b860 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:16.976759 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe9cf2-cf38-4e7d-8f65-57c93e23b860 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:16.978549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-20e98b9b-850b-487f-bc6e-c4dbce42a129 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:16.978549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:16.978549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:16.978549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:16.978549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:16.978549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:16.978549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:16.978549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:16.978549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:16.978549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:16.978549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:16.978549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:16.978549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 14:07:16.978549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 14:07:16.978549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 14:07:16.979524 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 14:07:16.979524 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:16.979524 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:16.979524 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:16.979524 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:16.979524 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:16.979524 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:16.979524 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:16.979524 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:16.979524 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:16.979524 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:16.979524 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:16.979524 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:16.979524 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:16.979524 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:16.979524 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:16.979524 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:16.980434 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 14:07:16.980434 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 14:07:16.980434 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:07:16.980434 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:07:16.980434 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:07:16.980434 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:07:16.980434 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:16.980434 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:16.980434 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 268/513] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:07:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:16.980956 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e98b9b-850b-487f-bc6e-c4dbce42a129 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:16.992693 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbe9cf2-cf38-4e7d-8f65-57c93e23b860 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:16.994133 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbe9cf2-cf38-4e7d-8f65-57c93e23b860 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:17.007985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-cdbe9cf2-cf38-4e7d-8f65-57c93e23b860 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:17.007985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:17.007985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:17.007985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:17.007985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:17.007985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:17.007985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:17.007985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:17.007985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:17.007985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:17.007985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:17.007985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:17.007985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:17.007985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:17.007985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:17.008884 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:17.008884 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:17.008884 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:17.008884 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:17.008884 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:17.008884 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:17.008884 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:17.008884 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:17.008884 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:17.008884 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:17.008884 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:17.008884 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:17.008884 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:17.008884 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:17.008884 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:17.008884 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:17.008884 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:17.009725 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:17.009725 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:17.009725 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:17.009725 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:17.009725 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 246/514] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:17.009998 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:17.010067 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe9cf2-cf38-4e7d-8f65-57c93e23b860 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:17.028001 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:17.085168 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e98b9b-850b-487f-bc6e-c4dbce42a129 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:17.108075 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe9cf2-cf38-4e7d-8f65-57c93e23b860 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:17.120057 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c70bb-f05d-485e-9b76-132045076cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:17.120708 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c70bb-f05d-485e-9b76-132045076cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:17.120989 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c70bb-f05d-485e-9b76-132045076cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:17.121260 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c70bb-f05d-485e-9b76-132045076cc4 None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:17.122196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c70bb-f05d-485e-9b76-132045076cc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:17.122824 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c70bb-f05d-485e-9b76-132045076cc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:17.123877 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c70bb-f05d-485e-9b76-132045076cc4 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:17.134744 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 269/515] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:17.135531 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c70bb-f05d-485e-9b76-132045076cc4 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:17.144213 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567c50d-d032-4ec6-8ddd-6332b7af754e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:17.144890 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567c50d-d032-4ec6-8ddd-6332b7af754e None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:17.145251 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567c50d-d032-4ec6-8ddd-6332b7af754e None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:17.145798 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567c50d-d032-4ec6-8ddd-6332b7af754e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:17.156340 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4567c50d-d032-4ec6-8ddd-6332b7af754e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:17.157798 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4567c50d-d032-4ec6-8ddd-6332b7af754e None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:17.172599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-4567c50d-d032-4ec6-8ddd-6332b7af754e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:17.172599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:17.172599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:17.172599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:17.172599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:17.172599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:17.172599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:17.172599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:17.172599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:17.172599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:17.172599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:17.172599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:17.172599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:17.172599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:17.172599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:17.173458 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:17.173458 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:17.173458 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:17.173458 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:17.173458 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:17.173458 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:17.173458 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:17.173458 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:17.173458 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:17.173458 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:17.173458 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:17.173458 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:17.173458 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:17.173458 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:17.173458 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:17.173458 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:17.173458 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:17.174592 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:17.174592 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:17.174592 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:17.174592 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:17.174592 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 247/516] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:17.175117 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567c50d-d032-4ec6-8ddd-6332b7af754e None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:17.226515 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:17.237724 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c70bb-f05d-485e-9b76-132045076cc4 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:17.239860 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:17.272004 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5625df2-f8fa-46e8-b079-2efbace4ca12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:17.272277 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567c50d-d032-4ec6-8ddd-6332b7af754e None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:17.272688 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5625df2-f8fa-46e8-b079-2efbace4ca12 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:17.272970 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5625df2-f8fa-46e8-b079-2efbace4ca12 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:17.273249 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5625df2-f8fa-46e8-b079-2efbace4ca12 None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:17.273933 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5625df2-f8fa-46e8-b079-2efbace4ca12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:17.274233 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5625df2-f8fa-46e8-b079-2efbace4ca12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:17.275584 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5625df2-f8fa-46e8-b079-2efbace4ca12 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:17.286826 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 270/517] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:17.287613 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5625df2-f8fa-46e8-b079-2efbace4ca12 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:17.307129 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f9dcc4-ab6a-4653-9208-7f7438906f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:17.307738 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f9dcc4-ab6a-4653-9208-7f7438906f5f None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:17.308016 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f9dcc4-ab6a-4653-9208-7f7438906f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:17.308281 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f9dcc4-ab6a-4653-9208-7f7438906f5f None admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:17.309046 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f9dcc4-ab6a-4653-9208-7f7438906f5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:17.309344 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f9dcc4-ab6a-4653-9208-7f7438906f5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:17.310743 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f9dcc4-ab6a-4653-9208-7f7438906f5f None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:17.328150 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 248/518] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Aug 5 14:07:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:17.328975 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f9dcc4-ab6a-4653-9208-7f7438906f5f None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:17.388697 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5625df2-f8fa-46e8-b079-2efbace4ca12 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:17.423457 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb1416-6c96-4667-8417-d92f3aef8e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:17.424084 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb1416-6c96-4667-8417-d92f3aef8e2e None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:17.424363 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb1416-6c96-4667-8417-d92f3aef8e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:17.424661 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb1416-6c96-4667-8417-d92f3aef8e2e None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:17.425342 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb1416-6c96-4667-8417-d92f3aef8e2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:17.425634 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb1416-6c96-4667-8417-d92f3aef8e2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:17.427099 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb1416-6c96-4667-8417-d92f3aef8e2e None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:17.431565 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f9dcc4-ab6a-4653-9208-7f7438906f5f None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:17.438759 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 271/519] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:17.439818 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb1416-6c96-4667-8417-d92f3aef8e2e None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:17.450173 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:17.459325 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:17.466869 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab1385c-8b68-418f-92f4-1a30ecf1db39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:17.467506 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab1385c-8b68-418f-92f4-1a30ecf1db39 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:17.467778 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab1385c-8b68-418f-92f4-1a30ecf1db39 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:17.468043 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab1385c-8b68-418f-92f4-1a30ecf1db39 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:17.475780 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab1385c-8b68-418f-92f4-1a30ecf1db39 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:17.477237 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab1385c-8b68-418f-92f4-1a30ecf1db39 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:17.489844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-9ab1385c-8b68-418f-92f4-1a30ecf1db39 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 14:07:17.489844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:17.489844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:17.489844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:17.489844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:17.489844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:17.489844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:17.489844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:17.489844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:17.489844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:17.489844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:17.489844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:17.489844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 14:07:17.489844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 14:07:17.489844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 14:07:17.489844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 14:07:17.490909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:17.490909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:17.490909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:07:17.490909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:17.490909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:07:17.490909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:17.490909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:17.490909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:17.490909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:17.490909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:17.490909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:17.490909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:17.490909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:17.490909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:17.490909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:17.490909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:17.490909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:17.490909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:17.491839 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:17.491839 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:17.491839 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 14:07:17.491839 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 14:07:17.491839 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 14:07:17.491839 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 14:07:17.491839 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 14:07:17.491839 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 14:07:17.491839 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 14:07:17.491839 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:17.491839 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 249/520] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:07:17 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:17.492429 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab1385c-8b68-418f-92f4-1a30ecf1db39 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:17.546588 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb1416-6c96-4667-8417-d92f3aef8e2e None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:17.580460 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ce39b-65a1-450f-a4fc-60f56e616ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:17.581068 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ce39b-65a1-450f-a4fc-60f56e616ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:17.581347 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ce39b-65a1-450f-a4fc-60f56e616ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:17.581621 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ce39b-65a1-450f-a4fc-60f56e616ed8 None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:17.582311 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2ce39b-65a1-450f-a4fc-60f56e616ed8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:17.583246 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2ce39b-65a1-450f-a4fc-60f56e616ed8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:17.584633 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2ce39b-65a1-450f-a4fc-60f56e616ed8 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:17.594647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab1385c-8b68-418f-92f4-1a30ecf1db39 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:17.595548 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 272/521] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:17.596375 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ce39b-65a1-450f-a4fc-60f56e616ed8 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:17.631381 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f220b200-0bfe-4834-a543-e40cd535c168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:17.632009 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f220b200-0bfe-4834-a543-e40cd535c168 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:17.632290 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f220b200-0bfe-4834-a543-e40cd535c168 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:17.632606 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f220b200-0bfe-4834-a543-e40cd535c168 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:17.640158 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f220b200-0bfe-4834-a543-e40cd535c168 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:17.641583 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f220b200-0bfe-4834-a543-e40cd535c168 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:17.653676 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-f220b200-0bfe-4834-a543-e40cd535c168 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 14:07:17.653676 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:17.653676 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:17.653676 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:17.653676 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:17.653676 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:17.653676 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:17.653676 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:17.653676 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:17.653676 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:17.653676 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:17.653676 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:17.653676 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 14:07:17.653676 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 14:07:17.653676 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 14:07:17.653676 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 14:07:17.654566 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:17.654566 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:17.654566 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:07:17.654566 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:17.654566 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:07:17.654566 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:17.654566 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:17.654566 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:17.654566 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:17.654566 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:17.654566 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:17.654566 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:17.654566 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:17.654566 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:17.654566 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:17.654566 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:17.654566 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:17.654566 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:17.655454 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:17.655454 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:17.655454 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 14:07:17.655454 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 14:07:17.655454 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 14:07:17.655454 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 14:07:17.655454 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 14:07:17.655454 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 14:07:17.655454 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 14:07:17.655454 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:17.655454 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 250/522] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Aug 5 14:07:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:17.656150 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f220b200-0bfe-4834-a543-e40cd535c168 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:17.667482 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:17.700223 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ce39b-65a1-450f-a4fc-60f56e616ed8 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:17.702431 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:17.734670 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f9d532-f549-470f-89e5-292f0e6213fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:17.735320 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9d532-f549-470f-89e5-292f0e6213fc None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:17.735595 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9d532-f549-470f-89e5-292f0e6213fc None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:17.735864 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9d532-f549-470f-89e5-292f0e6213fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:17.736678 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f9d532-f549-470f-89e5-292f0e6213fc None admin] RBAC: Adding query filter params (role.id=ec18a2850fd540a39d4a935348bb4474, scope.project.id=319973e011c54b4ca1f2198c8a5e135b, user.id=dd1ce2a2b4eb405b999d3fb756795176) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:17.736984 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f9d532-f549-470f-89e5-292f0e6213fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:17.738408 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f9d532-f549-470f-89e5-292f0e6213fc None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:17.743982 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 273/523] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:07:17 2022] GET /identity/v3/role_assignments?user.id=dd1ce2a2b4eb405b999d3fb756795176&scope.project.id=319973e011c54b4ca1f2198c8a5e135b&role.id=ec18a2850fd540a39d4a935348bb4474 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:17.745017 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f9d532-f549-470f-89e5-292f0e6213fc None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:17.763591 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f220b200-0bfe-4834-a543-e40cd535c168 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:17.805361 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642884f9-d811-429b-ac3f-98d27417192b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:17.805992 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642884f9-d811-429b-ac3f-98d27417192b None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:17.806265 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642884f9-d811-429b-ac3f-98d27417192b None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:17.806532 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642884f9-d811-429b-ac3f-98d27417192b None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:17.807211 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642884f9-d811-429b-ac3f-98d27417192b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:17.808676 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642884f9-d811-429b-ac3f-98d27417192b None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:17.853691 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f9d532-f549-470f-89e5-292f0e6213fc None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:17.885358 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:17.890246 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d626087d-e96a-4eaf-8f3f-a14905b7bf9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:17.890798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626087d-e96a-4eaf-8f3f-a14905b7bf9e None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:17.891102 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626087d-e96a-4eaf-8f3f-a14905b7bf9e None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:17.891343 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626087d-e96a-4eaf-8f3f-a14905b7bf9e None admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:17.892135 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d626087d-e96a-4eaf-8f3f-a14905b7bf9e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:17.892461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d626087d-e96a-4eaf-8f3f-a14905b7bf9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:17.893875 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d626087d-e96a-4eaf-8f3f-a14905b7bf9e None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:17.894900 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 251/524] 10.222.0.26 () {60 vars in 1132 bytes} [Fri Aug 5 14:07:17 2022] POST /identity/v3/users => generated 247 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:07:17.897886 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642884f9-d811-429b-ac3f-98d27417192b None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:17.928381 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 274/525] 10.222.0.26 () {56 vars in 1137 bytes} [Fri Aug 5 14:07:17 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:17.929703 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d626087d-e96a-4eaf-8f3f-a14905b7bf9e None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:18.034113 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642884f9-d811-429b-ac3f-98d27417192b None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:18.037962 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:18.078611 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f9b87-79cc-4b73-a88c-fa0d61514d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:18.079409 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f9b87-79cc-4b73-a88c-fa0d61514d5d None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:18.080471 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f9b87-79cc-4b73-a88c-fa0d61514d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:18.080471 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f9b87-79cc-4b73-a88c-fa0d61514d5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:18.080847 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d626087d-e96a-4eaf-8f3f-a14905b7bf9e None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:18.087701 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f9b87-79cc-4b73-a88c-fa0d61514d5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:18.089166 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f9b87-79cc-4b73-a88c-fa0d61514d5d None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:18.103812 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-a76f9b87-79cc-4b73-a88c-fa0d61514d5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:18.103812 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:18.103812 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:18.103812 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:18.103812 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:18.103812 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:18.103812 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:18.103812 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:18.103812 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:18.103812 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:18.103812 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:18.103812 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:18.103812 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 14:07:18.103812 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 14:07:18.103812 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 14:07:18.104764 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 14:07:18.104764 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:18.104764 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:18.104764 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:18.104764 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:18.104764 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:18.104764 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:18.104764 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:18.104764 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:18.104764 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:18.104764 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:18.104764 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:18.104764 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:18.104764 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:18.104764 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:18.104764 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:18.104764 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:18.105686 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 14:07:18.105686 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 14:07:18.105686 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:07:18.105686 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:07:18.105686 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:07:18.105686 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:07:18.105686 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:18.105686 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:18.109159 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 252/526] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:07:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:18.110045 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f9b87-79cc-4b73-a88c-fa0d61514d5d None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:18.137080 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9847659c-90de-4a49-868a-94a89dd492a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:18.137879 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9847659c-90de-4a49-868a-94a89dd492a5 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:18.138156 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9847659c-90de-4a49-868a-94a89dd492a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:18.138431 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9847659c-90de-4a49-868a-94a89dd492a5 None admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:18.139224 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9847659c-90de-4a49-868a-94a89dd492a5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:18.139527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9847659c-90de-4a49-868a-94a89dd492a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:18.141081 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9847659c-90de-4a49-868a-94a89dd492a5 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:18.167719 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 275/527] 10.222.0.26 () {56 vars in 1139 bytes} [Fri Aug 5 14:07:17 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:18.172843 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9847659c-90de-4a49-868a-94a89dd492a5 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:18.235467 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f9b87-79cc-4b73-a88c-fa0d61514d5d None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:18.256822 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:18.273463 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a3079d-596f-4ecf-abf1-872a1f9304e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:18.274300 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3079d-596f-4ecf-abf1-872a1f9304e4 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:18.274747 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3079d-596f-4ecf-abf1-872a1f9304e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:18.275178 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3079d-596f-4ecf-abf1-872a1f9304e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:18.310659 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:18.315177 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9847659c-90de-4a49-868a-94a89dd492a5 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:18.354736 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a3079d-596f-4ecf-abf1-872a1f9304e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:18.356518 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a3079d-596f-4ecf-abf1-872a1f9304e4 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:18.360506 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd03b2d-f8fd-478b-8eeb-1731cfda7816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:18.361210 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd03b2d-f8fd-478b-8eeb-1731cfda7816 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:18.361581 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd03b2d-f8fd-478b-8eeb-1731cfda7816 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:18.361925 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd03b2d-f8fd-478b-8eeb-1731cfda7816 None admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:18.362744 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd03b2d-f8fd-478b-8eeb-1731cfda7816 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:18.363118 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd03b2d-f8fd-478b-8eeb-1731cfda7816 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:18.364741 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd03b2d-f8fd-478b-8eeb-1731cfda7816 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:18.392571 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 276/528] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Aug 5 14:07:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:18.393746 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd03b2d-f8fd-478b-8eeb-1731cfda7816 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:18.475257 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 253/529] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:07:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 14:07:18.477072 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a3079d-596f-4ecf-abf1-872a1f9304e4 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:18.536406 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd03b2d-f8fd-478b-8eeb-1731cfda7816 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:18.554374 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:18.572784 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:18.587939 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c62ad8e-130c-4bfb-b238-cbb32ea20dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:18.588758 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62ad8e-130c-4bfb-b238-cbb32ea20dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:18.589144 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62ad8e-130c-4bfb-b238-cbb32ea20dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:18.589516 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62ad8e-130c-4bfb-b238-cbb32ea20dd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:18.619064 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a3079d-596f-4ecf-abf1-872a1f9304e4 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:18.655943 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68349d57-a7d4-4e46-813f-a200eac939a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:18.656926 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68349d57-a7d4-4e46-813f-a200eac939a6 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:18.657410 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68349d57-a7d4-4e46-813f-a200eac939a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:18.657895 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68349d57-a7d4-4e46-813f-a200eac939a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:18.658873 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68349d57-a7d4-4e46-813f-a200eac939a6 None admin] RBAC: Adding query filter params (role.id=ec18a2850fd540a39d4a935348bb4474, scope.project.id=319973e011c54b4ca1f2198c8a5e135b, user.id=5ec4940c99fc4fa8a5bc7cdabc4558fb) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:18.659381 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68349d57-a7d4-4e46-813f-a200eac939a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:18.661883 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68349d57-a7d4-4e46-813f-a200eac939a6 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:18.663497 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c62ad8e-130c-4bfb-b238-cbb32ea20dd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:18.665024 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c62ad8e-130c-4bfb-b238-cbb32ea20dd8 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:18.668546 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 254/530] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:07:18 2022] GET /identity/v3/role_assignments?user.id=5ec4940c99fc4fa8a5bc7cdabc4558fb&scope.project.id=319973e011c54b4ca1f2198c8a5e135b&role.id=ec18a2850fd540a39d4a935348bb4474 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:18.669694 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68349d57-a7d4-4e46-813f-a200eac939a6 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:18.791864 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68349d57-a7d4-4e46-813f-a200eac939a6 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:18.822441 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:18.833975 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff45319-130d-434b-991f-0e4503f23b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:18.837865 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff45319-130d-434b-991f-0e4503f23b98 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:18.837865 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff45319-130d-434b-991f-0e4503f23b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:18.837865 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff45319-130d-434b-991f-0e4503f23b98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:18.844428 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 277/531] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:07:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 14:07:18.851091 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c62ad8e-130c-4bfb-b238-cbb32ea20dd8 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:18.851305 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff45319-130d-434b-991f-0e4503f23b98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:18.854320 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff45319-130d-434b-991f-0e4503f23b98 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:18.862273 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:18.874291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-4ff45319-130d-434b-991f-0e4503f23b98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:18.874291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:18.874291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:18.874291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:18.874291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:18.874291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:18.874291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:18.874291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:18.874291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:18.874291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:18.874291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:18.874291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:18.874291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:18.874291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:18.874291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:18.878018 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:18.878018 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:18.878018 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:18.878018 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:18.878018 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:18.878018 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:18.878018 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:18.878018 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:18.878018 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:18.878018 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:18.878018 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:18.878018 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:18.878018 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:18.878018 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:18.878018 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:18.878018 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:18.878018 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:18.878865 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:18.878865 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:18.878865 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:18.878865 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:18.878865 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 255/532] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:18.881745 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff45319-130d-434b-991f-0e4503f23b98 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:19.017679 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff45319-130d-434b-991f-0e4503f23b98 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:19.056435 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064caf41-e1b2-45ad-a823-f4065fc05d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:19.057046 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c62ad8e-130c-4bfb-b238-cbb32ea20dd8 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:19.057158 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064caf41-e1b2-45ad-a823-f4065fc05d9f None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:19.057338 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064caf41-e1b2-45ad-a823-f4065fc05d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:19.057601 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064caf41-e1b2-45ad-a823-f4065fc05d9f None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:19.058292 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064caf41-e1b2-45ad-a823-f4065fc05d9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:19.058593 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064caf41-e1b2-45ad-a823-f4065fc05d9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:19.062378 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064caf41-e1b2-45ad-a823-f4065fc05d9f None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:19.081499 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 256/533] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:19.083447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064caf41-e1b2-45ad-a823-f4065fc05d9f None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:19.133759 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:19.136480 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe08340-b8c8-43df-9b7f-254076455898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:19.137369 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe08340-b8c8-43df-9b7f-254076455898 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:19.137911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe08340-b8c8-43df-9b7f-254076455898 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:19.138383 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe08340-b8c8-43df-9b7f-254076455898 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:19.166534 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe08340-b8c8-43df-9b7f-254076455898 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:19.168458 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe08340-b8c8-43df-9b7f-254076455898 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:19.196189 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064caf41-e1b2-45ad-a823-f4065fc05d9f None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:19.198999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-bbe08340-b8c8-43df-9b7f-254076455898 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:19.198999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:19.198999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:19.198999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:19.198999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:19.198999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:19.198999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:19.198999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:19.198999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:19.198999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:19.198999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:19.198999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:19.198999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:19.198999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:19.198999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:19.199838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:19.199838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:19.199838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:19.199838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:19.199838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:19.199838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:19.199838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:19.199838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:19.199838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:19.199838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:19.199838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:19.199838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:19.199838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:19.199838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:19.199838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:19.199838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:19.199838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:19.200696 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:19.200696 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:19.200696 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:19.200696 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:19.208900 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 278/534] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:07:19.210123 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe08340-b8c8-43df-9b7f-254076455898 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:19.239433 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8dae46-e804-43e0-afd7-6409be466456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:19.239433 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8dae46-e804-43e0-afd7-6409be466456 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:19.239433 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8dae46-e804-43e0-afd7-6409be466456 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:19.239817 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8dae46-e804-43e0-afd7-6409be466456 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:19.247167 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8dae46-e804-43e0-afd7-6409be466456 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:19.248640 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8dae46-e804-43e0-afd7-6409be466456 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:19.261923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-3c8dae46-e804-43e0-afd7-6409be466456 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:19.261923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:19.261923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:19.261923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:19.261923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:19.261923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:19.261923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:19.261923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:19.261923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:19.261923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:19.261923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:19.261923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:19.261923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 14:07:19.261923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 14:07:19.261923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 14:07:19.262852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 14:07:19.262852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:19.262852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:19.262852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:19.262852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:19.262852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:19.262852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:19.262852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:19.262852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:19.262852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:19.262852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:19.262852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:19.262852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:19.262852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:19.262852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:19.262852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:19.262852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:19.263707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 14:07:19.263707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 14:07:19.263707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:07:19.263707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:07:19.263707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:07:19.263707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:07:19.263707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:19.263707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:19.268082 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 257/535] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:07:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:19.269539 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8dae46-e804-43e0-afd7-6409be466456 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:19.368442 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:19.433574 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe08340-b8c8-43df-9b7f-254076455898 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:19.479718 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51be78-9efc-4c3e-98f7-7abdcd9abd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:19.480365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51be78-9efc-4c3e-98f7-7abdcd9abd82 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:19.481677 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51be78-9efc-4c3e-98f7-7abdcd9abd82 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:19.481963 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51be78-9efc-4c3e-98f7-7abdcd9abd82 None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:19.483611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be51be78-9efc-4c3e-98f7-7abdcd9abd82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:19.483923 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be51be78-9efc-4c3e-98f7-7abdcd9abd82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:19.485351 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be51be78-9efc-4c3e-98f7-7abdcd9abd82 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:19.488214 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:19.492588 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8dae46-e804-43e0-afd7-6409be466456 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:19.518665 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 279/536] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:19.530584 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51be78-9efc-4c3e-98f7-7abdcd9abd82 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:19.579650 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16c18c-02bb-4751-ba67-8ddb47277087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:19.580283 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16c18c-02bb-4751-ba67-8ddb47277087 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:19.580588 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16c18c-02bb-4751-ba67-8ddb47277087 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:19.580862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16c18c-02bb-4751-ba67-8ddb47277087 None admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:19.581615 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16c18c-02bb-4751-ba67-8ddb47277087 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:19.581922 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16c18c-02bb-4751-ba67-8ddb47277087 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:19.589209 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16c18c-02bb-4751-ba67-8ddb47277087 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:19.614454 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 258/537] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Aug 5 14:07:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:19.620461 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16c18c-02bb-4751-ba67-8ddb47277087 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:19.645796 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51be78-9efc-4c3e-98f7-7abdcd9abd82 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:19.669540 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:19.687992 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc313a-5df9-49ef-be95-8ba7447cbcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:19.689495 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc313a-5df9-49ef-be95-8ba7447cbcfc None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:19.690443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc313a-5df9-49ef-be95-8ba7447cbcfc None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:19.690725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc313a-5df9-49ef-be95-8ba7447cbcfc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:19.710625 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cc313a-5df9-49ef-be95-8ba7447cbcfc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:19.712094 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cc313a-5df9-49ef-be95-8ba7447cbcfc None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:19.731257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-69cc313a-5df9-49ef-be95-8ba7447cbcfc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:19.731257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:19.731257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:19.731257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:19.731257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:19.731257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:19.731257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:19.731257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:19.731257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:19.731257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:19.731257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:19.731257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:19.731257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 14:07:19.731257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 14:07:19.731257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 14:07:19.732191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 14:07:19.732191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:19.732191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:19.732191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:19.732191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:19.732191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:19.732191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:19.732191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:19.732191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:19.732191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:19.732191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:19.732191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:19.732191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:19.732191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:19.732191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:19.732191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:19.732191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:19.733899 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 14:07:19.733899 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 14:07:19.733899 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:07:19.733899 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:07:19.733899 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:07:19.733899 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:07:19.733899 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:19.733899 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:19.733899 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 280/538] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:07:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:19.741092 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc313a-5df9-49ef-be95-8ba7447cbcfc None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:19.826886 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16c18c-02bb-4751-ba67-8ddb47277087 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:19.893492 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f70f4d-220a-429f-b322-ef36d5508ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:19.894116 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f70f4d-220a-429f-b322-ef36d5508ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:19.894394 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f70f4d-220a-429f-b322-ef36d5508ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:19.894667 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f70f4d-220a-429f-b322-ef36d5508ec0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:19.895565 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f70f4d-220a-429f-b322-ef36d5508ec0 None admin] RBAC: Adding query filter params (role.id=ec18a2850fd540a39d4a935348bb4474, scope.project.id=319973e011c54b4ca1f2198c8a5e135b, user.id=88d6a1bae9834e2e91d4ab3a4c00959b) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:19.895932 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f70f4d-220a-429f-b322-ef36d5508ec0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:19.897481 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f70f4d-220a-429f-b322-ef36d5508ec0 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:19.905692 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc313a-5df9-49ef-be95-8ba7447cbcfc None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:19.914678 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 259/539] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:07:19 2022] GET /identity/v3/role_assignments?user.id=88d6a1bae9834e2e91d4ab3a4c00959b&scope.project.id=319973e011c54b4ca1f2198c8a5e135b&role.id=ec18a2850fd540a39d4a935348bb4474 => generated 259 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:19.954368 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6566d0-1b57-42e7-a7e4-cc762123eedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:19.955014 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6566d0-1b57-42e7-a7e4-cc762123eedf None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:19.955300 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6566d0-1b57-42e7-a7e4-cc762123eedf None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:19.955569 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6566d0-1b57-42e7-a7e4-cc762123eedf None admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:19.956333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6566d0-1b57-42e7-a7e4-cc762123eedf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:19.956698 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6566d0-1b57-42e7-a7e4-cc762123eedf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:19.958214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6566d0-1b57-42e7-a7e4-cc762123eedf None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:19.981105 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 281/540] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Aug 5 14:07:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:19.987428 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f70f4d-220a-429f-b322-ef36d5508ec0 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:20.016440 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25497d8-2f28-4f9b-ae2b-43c71162dc97 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:20.017071 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25497d8-2f28-4f9b-ae2b-43c71162dc97 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:20.017551 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25497d8-2f28-4f9b-ae2b-43c71162dc97 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:20.021256 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 282/541] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:20.026090 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5eab5-0580-4612-a85c-e01753be0803 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:20.026683 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5eab5-0580-4612-a85c-e01753be0803 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:20.027150 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5eab5-0580-4612-a85c-e01753be0803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:20.044901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:20.064993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-52f5eab5-0580-4612-a85c-e01753be0803 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:20.075606 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-52f5eab5-0580-4612-a85c-e01753be0803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:20.140994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:20.151305 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f70f4d-220a-429f-b322-ef36d5508ec0 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:20.200575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 283/542] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 14:07:20.203318 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c572801f-0a16-406c-86af-aba469bbcb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:20.204209 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572801f-0a16-406c-86af-aba469bbcb2d None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:20.204721 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572801f-0a16-406c-86af-aba469bbcb2d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:20.205196 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572801f-0a16-406c-86af-aba469bbcb2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:20.206190 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c572801f-0a16-406c-86af-aba469bbcb2d None admin] RBAC: Adding query filter params (role.id=ec18a2850fd540a39d4a935348bb4474, scope.project.id=319973e011c54b4ca1f2198c8a5e135b, user.id=695e37560f8f495e987d1bd17acc2680) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:20.206704 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c572801f-0a16-406c-86af-aba469bbcb2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:20.208440 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c572801f-0a16-406c-86af-aba469bbcb2d None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:20.214867 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 260/543] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:07:19 2022] GET /identity/v3/role_assignments?user.id=695e37560f8f495e987d1bd17acc2680&scope.project.id=319973e011c54b4ca1f2198c8a5e135b&role.id=ec18a2850fd540a39d4a935348bb4474 => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:20.251868 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f5eab5-0580-4612-a85c-e01753be0803 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:20.370243 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f5eab5-0580-4612-a85c-e01753be0803 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:20.408815 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c476ed-9d50-45d1-958b-e69ae7919feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:20.409842 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c476ed-9d50-45d1-958b-e69ae7919feb None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:20.410327 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c476ed-9d50-45d1-958b-e69ae7919feb None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:20.410919 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c476ed-9d50-45d1-958b-e69ae7919feb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:20.423206 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c476ed-9d50-45d1-958b-e69ae7919feb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:07:20.428955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-d6c476ed-9d50-45d1-958b-e69ae7919feb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:07:20.428955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:20.428955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:20.428955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:20.428955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:20.428955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:20.428955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:20.428955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:20.428955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:20.428955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:20.428955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:20.428955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:20.428955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:07:20.428955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:07:20.428955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:07:20.428955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:07:20.429820 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:07:20.429820 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:07:20.429820 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:07:20.429820 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:07:20.429820 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:20.429820 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:20.429820 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:20.429820 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:20.429820 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:20.429820 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:20.429820 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:20.429820 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:20.429820 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:20.429820 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:20.429820 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:20.429820 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:20.430747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:20.430747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:20.430747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:20.430747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:20.430747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:07:20.430747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:07:20.430747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:07:20.430747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:07:20.430747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:07:20.430747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:07:20.430747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:07:20.430747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:20.430747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 284/544] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Aug 5 14:07:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:20.438338 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c572801f-0a16-406c-86af-aba469bbcb2d None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:20.568440 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:20.581040 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c572801f-0a16-406c-86af-aba469bbcb2d None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:20.635563 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80b7ea-f140-4af0-86bf-f32b50ceb4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:20.636571 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80b7ea-f140-4af0-86bf-f32b50ceb4d4 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:20.637071 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80b7ea-f140-4af0-86bf-f32b50ceb4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:20.637549 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80b7ea-f140-4af0-86bf-f32b50ceb4d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:20.638476 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a80b7ea-f140-4af0-86bf-f32b50ceb4d4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:20.638974 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a80b7ea-f140-4af0-86bf-f32b50ceb4d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:20.640566 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a80b7ea-f140-4af0-86bf-f32b50ceb4d4 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:20.652632 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 261/545] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Aug 5 14:07:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:20.657826 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c476ed-9d50-45d1-958b-e69ae7919feb None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:20.695391 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22803ccf-78c3-430f-b993-d870a7dda6fb None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:20.695992 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22803ccf-78c3-430f-b993-d870a7dda6fb None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:20.696488 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22803ccf-78c3-430f-b993-d870a7dda6fb None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:20.697329 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:20.698491 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 262/546] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:07:20.703371 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4632d-c844-484c-98b8-f9f44d5331fd None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:20.711432 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4632d-c844-484c-98b8-f9f44d5331fd None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:20.711432 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4632d-c844-484c-98b8-f9f44d5331fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:20.741434 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-0fe4632d-c844-484c-98b8-f9f44d5331fd None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:20.742968 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-0fe4632d-c844-484c-98b8-f9f44d5331fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:20.776591 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c476ed-9d50-45d1-958b-e69ae7919feb None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:20.813913 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a443b-54ca-4ed8-b502-b522360d20de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:20.814797 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a443b-54ca-4ed8-b502-b522360d20de None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:20.815083 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a443b-54ca-4ed8-b502-b522360d20de None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:20.815389 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a443b-54ca-4ed8-b502-b522360d20de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:20.827307 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452a443b-54ca-4ed8-b502-b522360d20de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:20.828929 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452a443b-54ca-4ed8-b502-b522360d20de None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:20.845691 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-452a443b-54ca-4ed8-b502-b522360d20de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:20.845691 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:20.845691 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:20.845691 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:20.845691 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:20.845691 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:20.845691 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:20.845691 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:20.845691 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:20.845691 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:20.845691 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:20.845691 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:20.845691 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:20.845691 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:20.845691 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:20.846575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:20.846575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:20.846575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:20.846575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:20.846575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:20.846575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:20.846575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:20.846575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:20.846575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:20.846575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:20.846575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:20.846575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:20.846575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:20.846575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:20.846575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:20.846575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:20.846575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:20.847493 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:20.847493 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:20.847493 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:20.847493 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:20.847493 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 285/547] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:20.852991 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a443b-54ca-4ed8-b502-b522360d20de None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:20.893940 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 263/548] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:07:20.933671 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4632d-c844-484c-98b8-f9f44d5331fd None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:20.979748 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:21.018117 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a443b-54ca-4ed8-b502-b522360d20de None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:21.076603 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:21.090276 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ca5b7-eebb-4c35-84cf-533c7f37b697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:21.096993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ca5b7-eebb-4c35-84cf-533c7f37b697 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:21.096993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ca5b7-eebb-4c35-84cf-533c7f37b697 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:21.096993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ca5b7-eebb-4c35-84cf-533c7f37b697 None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:21.096993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2ca5b7-eebb-4c35-84cf-533c7f37b697 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:21.096993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2ca5b7-eebb-4c35-84cf-533c7f37b697 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:21.096993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2ca5b7-eebb-4c35-84cf-533c7f37b697 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:21.113000 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4632d-c844-484c-98b8-f9f44d5331fd None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:21.120042 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 286/549] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:21.124140 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ca5b7-eebb-4c35-84cf-533c7f37b697 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:21.180643 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aa3bea-5df3-4e7b-ab3a-a9f25160e64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:21.181322 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa3bea-5df3-4e7b-ab3a-a9f25160e64d None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:21.181566 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa3bea-5df3-4e7b-ab3a-a9f25160e64d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:21.181863 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa3bea-5df3-4e7b-ab3a-a9f25160e64d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:21.202924 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aa3bea-5df3-4e7b-ab3a-a9f25160e64d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:07:21.209966 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-04aa3bea-5df3-4e7b-ab3a-a9f25160e64d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:07:21.209966 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:21.209966 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:21.209966 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:21.209966 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:21.209966 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:21.209966 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:21.209966 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:21.209966 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:21.209966 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:21.209966 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:21.209966 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:21.209966 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:07:21.209966 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:07:21.209966 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:07:21.209966 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:07:21.211069 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:07:21.211069 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:07:21.211069 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:07:21.211069 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:07:21.211069 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:21.211069 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:21.211069 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:21.211069 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:21.211069 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:21.211069 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:21.211069 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:21.211069 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:21.211069 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:21.211069 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:21.211069 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:21.211069 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:21.211874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:21.211874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:21.211874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:21.211874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:21.211874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:07:21.211874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:07:21.211874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:07:21.211874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:07:21.211874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:07:21.211874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:07:21.211874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:07:21.211874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:21.214862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 264/550] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Aug 5 14:07:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:21.216711 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809234d-9d0c-47af-bf78-844f3a9f8c33 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:21.217201 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809234d-9d0c-47af-bf78-844f3a9f8c33 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:21.217648 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809234d-9d0c-47af-bf78-844f3a9f8c33 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:21.219469 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 265/551] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:21.220387 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8809234d-9d0c-47af-bf78-844f3a9f8c33 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:21.278913 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ca5b7-eebb-4c35-84cf-533c7f37b697 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:21.324814 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d177a66-94fc-4ec7-a460-f0980deb3cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:21.326275 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d177a66-94fc-4ec7-a460-f0980deb3cbe None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:21.327071 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d177a66-94fc-4ec7-a460-f0980deb3cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:21.327714 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d177a66-94fc-4ec7-a460-f0980deb3cbe None admin] PATH_INFO: `/v3/users/glance` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:21.338314 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d177a66-94fc-4ec7-a460-f0980deb3cbe None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:21.340693 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d177a66-94fc-4ec7-a460-f0980deb3cbe None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:21.347368 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8809234d-9d0c-47af-bf78-844f3a9f8c33 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:21.356496 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-6d177a66-94fc-4ec7-a460-f0980deb3cbe None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 05 14:07:21.356496 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:21.356496 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:21.356496 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:21.356496 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:21.356496 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:21.356496 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:21.356496 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:21.356496 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:21.356496 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:21.356496 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:21.356496 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:21.356496 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 14:07:21.356496 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 14:07:21.356496 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 14:07:21.356496 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 14:07:21.357644 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:21.357644 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:21.357644 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:07:21.357644 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:21.357644 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:07:21.357644 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:21.357644 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:21.357644 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:21.357644 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:21.357644 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:21.357644 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:21.357644 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:21.357644 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:21.357644 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:21.357644 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:21.357644 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:21.357644 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:21.357644 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:21.358606 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:21.358606 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:21.358606 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 14:07:21.358606 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 14:07:21.358606 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 14:07:21.358606 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 14:07:21.358606 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 14:07:21.358606 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 14:07:21.358606 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 05 14:07:21.358606 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:21.361972 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 287/552] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:07:21 2022] GET /identity/v3/users/glance => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:07:21.363340 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f7f42-4807-44d3-a5e1-36a7b649b619 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:21.364054 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f7f42-4807-44d3-a5e1-36a7b649b619 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:21.364556 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f7f42-4807-44d3-a5e1-36a7b649b619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:21.372683 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:21.389278 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-486f7f42-4807-44d3-a5e1-36a7b649b619 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:21.393262 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-486f7f42-4807-44d3-a5e1-36a7b649b619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:21.395542 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e1c3d-ad8c-4cc2-80ef-1d0bad6b439d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:21.396237 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e1c3d-ad8c-4cc2-80ef-1d0bad6b439d None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:21.396560 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e1c3d-ad8c-4cc2-80ef-1d0bad6b439d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:21.396832 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e1c3d-ad8c-4cc2-80ef-1d0bad6b439d None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:21.397541 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e1c3d-ad8c-4cc2-80ef-1d0bad6b439d None admin] RBAC: Adding query filter params (name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:21.397844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e1c3d-ad8c-4cc2-80ef-1d0bad6b439d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:21.399200 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e1c3d-ad8c-4cc2-80ef-1d0bad6b439d None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:21.409554 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 266/553] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Aug 5 14:07:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:21.411140 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e1c3d-ad8c-4cc2-80ef-1d0bad6b439d None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:21.413785 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:21.488564 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 288/554] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:07:21.489944 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486f7f42-4807-44d3-a5e1-36a7b649b619 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:21.544159 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e1c3d-ad8c-4cc2-80ef-1d0bad6b439d None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:21.591080 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3ed60-8af3-4393-b28a-ca353a3b67eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:21.592552 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3ed60-8af3-4393-b28a-ca353a3b67eb None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:21.593251 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3ed60-8af3-4393-b28a-ca353a3b67eb None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:21.593675 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3ed60-8af3-4393-b28a-ca353a3b67eb None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:21.594874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad3ed60-8af3-4393-b28a-ca353a3b67eb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:21.595715 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad3ed60-8af3-4393-b28a-ca353a3b67eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:21.597330 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad3ed60-8af3-4393-b28a-ca353a3b67eb None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:21.611626 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486f7f42-4807-44d3-a5e1-36a7b649b619 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:21.627200 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 267/555] 10.222.0.26 () {56 vars in 1137 bytes} [Fri Aug 5 14:07:21 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:21.627200 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3ed60-8af3-4393-b28a-ca353a3b67eb None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:21.655371 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:21.660764 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c10ad5-05a2-4b42-a3ab-a7483551be59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:21.661599 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c10ad5-05a2-4b42-a3ab-a7483551be59 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:21.661679 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c10ad5-05a2-4b42-a3ab-a7483551be59 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:21.661938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c10ad5-05a2-4b42-a3ab-a7483551be59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:21.672732 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c10ad5-05a2-4b42-a3ab-a7483551be59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:21.674174 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c10ad5-05a2-4b42-a3ab-a7483551be59 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:21.690624 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-40c10ad5-05a2-4b42-a3ab-a7483551be59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:21.690624 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:21.690624 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:21.690624 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:21.690624 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:21.690624 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:21.690624 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:21.690624 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:21.690624 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:21.690624 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:21.690624 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:21.690624 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:21.690624 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:21.690624 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:21.690624 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:21.692124 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:21.692124 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:21.692124 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:21.692124 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:21.692124 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:21.692124 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:21.692124 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:21.692124 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:21.692124 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:21.692124 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:21.692124 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:21.692124 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:21.692124 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:21.692124 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:21.692124 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:21.692124 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:21.692124 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:21.693065 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:21.693065 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:21.693065 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:21.693065 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:21.693065 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 289/556] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:21.693828 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c10ad5-05a2-4b42-a3ab-a7483551be59 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:21.712599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:21.772325 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3ed60-8af3-4393-b28a-ca353a3b67eb None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:21.807472 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c10ad5-05a2-4b42-a3ab-a7483551be59 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:21.819267 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055d388-4e9c-4c8a-b9e0-deb79828181f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:21.820989 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055d388-4e9c-4c8a-b9e0-deb79828181f None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:21.821489 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055d388-4e9c-4c8a-b9e0-deb79828181f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:21.821956 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055d388-4e9c-4c8a-b9e0-deb79828181f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:21.840497 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0055d388-4e9c-4c8a-b9e0-deb79828181f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:07:21.845641 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d4a40-fc17-4e36-b255-082438a41e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:21.846284 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d4a40-fc17-4e36-b255-082438a41e99 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:21.846560 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d4a40-fc17-4e36-b255-082438a41e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:21.846833 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d4a40-fc17-4e36-b255-082438a41e99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:21.848585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-0055d388-4e9c-4c8a-b9e0-deb79828181f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:07:21.848585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:21.848585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:21.848585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:21.848585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:21.848585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:21.848585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:21.848585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:21.848585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:21.848585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:21.848585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:21.848585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:21.848585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:07:21.848585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:07:21.848585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:07:21.848585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:07:21.849475 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:07:21.849475 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:07:21.849475 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:07:21.849475 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:07:21.849475 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:21.849475 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:21.849475 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:21.849475 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:21.849475 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:21.849475 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:21.849475 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:21.849475 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:21.849475 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:21.849475 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:21.849475 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:21.849475 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:21.850781 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:21.850781 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:21.850781 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:21.850781 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:21.850781 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:07:21.850781 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:07:21.850781 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:07:21.850781 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:07:21.850781 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:07:21.850781 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:07:21.850781 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:07:21.850781 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:21.854180 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 268/557] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Aug 5 14:07:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:21.855358 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055d388-4e9c-4c8a-b9e0-deb79828181f None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:21.866952 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d4a40-fc17-4e36-b255-082438a41e99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:21.868398 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d4a40-fc17-4e36-b255-082438a41e99 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:21.883810 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-777d4a40-fc17-4e36-b255-082438a41e99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:21.883810 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:21.883810 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:21.883810 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:21.883810 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:21.883810 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:21.883810 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:21.883810 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:21.883810 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:21.883810 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:21.883810 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:21.883810 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:21.883810 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:21.883810 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:21.883810 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:21.884789 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:21.884789 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:21.884789 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:21.884789 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:21.884789 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:21.884789 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:21.884789 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:21.884789 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:21.884789 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:21.884789 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:21.884789 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:21.884789 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:21.884789 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:21.884789 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:21.884789 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:21.884789 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:21.884789 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:21.885692 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:21.885692 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:21.885692 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:21.885692 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:21.885692 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 290/558] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:21.885964 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d4a40-fc17-4e36-b255-082438a41e99 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:21.925743 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:21.995253 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055d388-4e9c-4c8a-b9e0-deb79828181f None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:22.002436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d4a40-fc17-4e36-b255-082438a41e99 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:22.017183 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:22.031599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d151953-26c4-4f6e-a5fe-16593dfca0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:22.032386 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d151953-26c4-4f6e-a5fe-16593dfca0a7 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:22.032545 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d151953-26c4-4f6e-a5fe-16593dfca0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:22.032794 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d151953-26c4-4f6e-a5fe-16593dfca0a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:22.033492 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d151953-26c4-4f6e-a5fe-16593dfca0a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:22.033799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d151953-26c4-4f6e-a5fe-16593dfca0a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:22.035298 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d151953-26c4-4f6e-a5fe-16593dfca0a7 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:22.050015 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fcb9c2-040d-44fa-ab2b-9a697701b193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:22.051525 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcb9c2-040d-44fa-ab2b-9a697701b193 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:22.052026 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcb9c2-040d-44fa-ab2b-9a697701b193 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:22.053027 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcb9c2-040d-44fa-ab2b-9a697701b193 None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:22.053225 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 269/559] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:22.054386 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fcb9c2-040d-44fa-ab2b-9a697701b193 None admin] RBAC: Adding query filter params (name=service) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:22.054656 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d151953-26c4-4f6e-a5fe-16593dfca0a7 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:22.054983 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fcb9c2-040d-44fa-ab2b-9a697701b193 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:22.056810 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fcb9c2-040d-44fa-ab2b-9a697701b193 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:22.068815 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 291/560] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Aug 5 14:07:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:22.069859 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fcb9c2-040d-44fa-ab2b-9a697701b193 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:22.172127 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d151953-26c4-4f6e-a5fe-16593dfca0a7 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:22.193210 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:22.197746 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fcb9c2-040d-44fa-ab2b-9a697701b193 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:22.209729 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee224119-4144-47b0-bfcd-e0802022fee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:22.210429 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee224119-4144-47b0-bfcd-e0802022fee7 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:22.210630 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee224119-4144-47b0-bfcd-e0802022fee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:22.210890 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee224119-4144-47b0-bfcd-e0802022fee7 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:22.211579 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee224119-4144-47b0-bfcd-e0802022fee7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:22.211880 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee224119-4144-47b0-bfcd-e0802022fee7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:22.213277 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee224119-4144-47b0-bfcd-e0802022fee7 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:22.230501 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 270/561] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:22.230501 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee224119-4144-47b0-bfcd-e0802022fee7 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:22.236805 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116aa0c6-5b0a-4876-9f32-73059a788a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:22.237477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116aa0c6-5b0a-4876-9f32-73059a788a81 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:22.237756 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116aa0c6-5b0a-4876-9f32-73059a788a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:22.238027 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116aa0c6-5b0a-4876-9f32-73059a788a81 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:22.246071 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116aa0c6-5b0a-4876-9f32-73059a788a81 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:22.247776 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116aa0c6-5b0a-4876-9f32-73059a788a81 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:22.260050 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-116aa0c6-5b0a-4876-9f32-73059a788a81 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 14:07:22.260050 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:22.260050 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:22.260050 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:22.260050 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:22.260050 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:22.260050 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:22.260050 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:22.260050 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:22.260050 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:22.260050 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:22.260050 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:22.260050 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 14:07:22.260050 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 14:07:22.260050 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 14:07:22.260050 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 14:07:22.261244 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:22.261244 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:22.261244 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:07:22.261244 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:22.261244 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:07:22.261244 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:22.261244 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:22.261244 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:22.261244 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:22.261244 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:22.261244 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:22.261244 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:22.261244 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:22.261244 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:22.261244 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:22.261244 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:22.261244 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:22.261244 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:22.262201 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:22.262201 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:22.262201 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 14:07:22.262201 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 14:07:22.262201 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 14:07:22.262201 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 14:07:22.262201 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 14:07:22.262201 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 14:07:22.262201 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 14:07:22.262201 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:22.262201 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 292/562] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Aug 5 14:07:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:22.262802 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9cf1ff-0b58-423d-8e06-8183bf20c36f None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:22.262802 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9cf1ff-0b58-423d-8e06-8183bf20c36f None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:22.262932 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9cf1ff-0b58-423d-8e06-8183bf20c36f None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:22.264126 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 293/563] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:22.269007 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9cf1ff-0b58-423d-8e06-8183bf20c36f None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:22.280517 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:22.341412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee224119-4144-47b0-bfcd-e0802022fee7 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:22.376124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9375fd-7bfe-4ff2-b6d2-d9befc0b5dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:22.376356 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9cf1ff-0b58-423d-8e06-8183bf20c36f None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:22.377111 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9375fd-7bfe-4ff2-b6d2-d9befc0b5dbb None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:22.377502 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9375fd-7bfe-4ff2-b6d2-d9befc0b5dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:22.377884 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9375fd-7bfe-4ff2-b6d2-d9befc0b5dbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:22.389698 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9375fd-7bfe-4ff2-b6d2-d9befc0b5dbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:22.391281 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9375fd-7bfe-4ff2-b6d2-d9befc0b5dbb None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:22.407102 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-7c9375fd-7bfe-4ff2-b6d2-d9befc0b5dbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:22.407102 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:22.407102 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:22.407102 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:22.407102 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:22.407102 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:22.407102 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:22.407102 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:22.407102 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:22.407102 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:22.407102 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:22.407102 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:22.407102 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:22.407102 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:22.407102 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:22.408230 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:22.408230 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:22.408230 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:22.408230 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:22.408230 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:22.408230 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:22.408230 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:22.408230 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:22.408230 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:22.408230 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:22.408230 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:22.408230 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:22.408230 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:22.408230 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:22.408230 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:22.408230 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:22.408230 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:22.409412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:22.409412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:22.409412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:22.409412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:22.410043 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 271/564] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:22.410790 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9375fd-7bfe-4ff2-b6d2-d9befc0b5dbb None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:22.419475 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce5c001-76da-4328-af1f-6c09173cebbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:22.420119 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5c001-76da-4328-af1f-6c09173cebbd None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:22.420399 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5c001-76da-4328-af1f-6c09173cebbd None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:22.420703 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5c001-76da-4328-af1f-6c09173cebbd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:22.428015 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce5c001-76da-4328-af1f-6c09173cebbd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:22.429473 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce5c001-76da-4328-af1f-6c09173cebbd None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:22.440760 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-6ce5c001-76da-4328-af1f-6c09173cebbd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:22.440760 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:22.440760 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:22.440760 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:22.440760 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:22.440760 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:22.440760 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:22.440760 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:22.440760 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:22.440760 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:22.440760 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:22.440760 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:22.440760 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 14:07:22.440760 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 14:07:22.440760 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 14:07:22.442251 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 14:07:22.442251 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:22.442251 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:22.442251 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:22.442251 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:22.442251 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:22.442251 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:22.442251 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:22.442251 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:22.442251 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:22.442251 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:22.442251 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:22.442251 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:22.442251 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:22.442251 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:22.442251 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:22.442251 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:22.443249 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 14:07:22.443249 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 14:07:22.443249 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:07:22.443249 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:07:22.443249 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:07:22.443249 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:07:22.443249 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:22.443249 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:22.443249 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 294/565] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:07:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:22.444190 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff53664-a9e1-453e-b816-1e4d65b81086 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:22.444190 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff53664-a9e1-453e-b816-1e4d65b81086 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:22.444477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff53664-a9e1-453e-b816-1e4d65b81086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:22.463282 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-aff53664-a9e1-453e-b816-1e4d65b81086 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:22.464514 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-aff53664-a9e1-453e-b816-1e4d65b81086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:22.478416 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:22.518000 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:22.532727 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9375fd-7bfe-4ff2-b6d2-d9befc0b5dbb None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:22.540477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 295/566] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:07:22.546411 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff53664-a9e1-453e-b816-1e4d65b81086 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:22.573105 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411df56e-f50b-4809-82a8-127f3a790c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:22.573719 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411df56e-f50b-4809-82a8-127f3a790c4b None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:22.577263 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411df56e-f50b-4809-82a8-127f3a790c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:22.577263 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411df56e-f50b-4809-82a8-127f3a790c4b None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:22.577263 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411df56e-f50b-4809-82a8-127f3a790c4b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:22.577263 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411df56e-f50b-4809-82a8-127f3a790c4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:22.577263 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411df56e-f50b-4809-82a8-127f3a790c4b None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:22.595406 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 272/567] 10.222.0.26 () {56 vars in 1135 bytes} [Fri Aug 5 14:07:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:22.595824 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411df56e-f50b-4809-82a8-127f3a790c4b None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:22.648721 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff53664-a9e1-453e-b816-1e4d65b81086 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:22.683387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e4c6b6-3f23-436e-afff-4ccee19d38e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:22.683996 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4c6b6-3f23-436e-afff-4ccee19d38e5 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:22.684250 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4c6b6-3f23-436e-afff-4ccee19d38e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:22.684550 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4c6b6-3f23-436e-afff-4ccee19d38e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:22.685232 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e4c6b6-3f23-436e-afff-4ccee19d38e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:22.685531 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e4c6b6-3f23-436e-afff-4ccee19d38e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:22.686870 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e4c6b6-3f23-436e-afff-4ccee19d38e5 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:22.697618 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 296/568] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:22.697754 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411df56e-f50b-4809-82a8-127f3a790c4b None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:22.698770 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df50f017-bbbf-41e1-8e29-d52cb731eb3f None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:22.699162 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df50f017-bbbf-41e1-8e29-d52cb731eb3f None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:22.699431 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df50f017-bbbf-41e1-8e29-d52cb731eb3f None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:22.702476 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 297/569] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:22.702476 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df50f017-bbbf-41e1-8e29-d52cb731eb3f None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:22.712871 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:22.732976 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d954a-eec8-4be3-87bb-0201afdd8654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:22.733587 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d954a-eec8-4be3-87bb-0201afdd8654 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:22.733891 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d954a-eec8-4be3-87bb-0201afdd8654 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:22.734164 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d954a-eec8-4be3-87bb-0201afdd8654 None admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:22.734899 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1d954a-eec8-4be3-87bb-0201afdd8654 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:22.735206 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1d954a-eec8-4be3-87bb-0201afdd8654 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:22.736668 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1d954a-eec8-4be3-87bb-0201afdd8654 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:22.748726 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 273/570] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Aug 5 14:07:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:22.749517 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d954a-eec8-4be3-87bb-0201afdd8654 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:22.767856 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:22.810290 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df50f017-bbbf-41e1-8e29-d52cb731eb3f None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:22.843671 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f57947-6618-4b33-9338-023ec6783b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:22.844307 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f57947-6618-4b33-9338-023ec6783b85 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:22.844622 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f57947-6618-4b33-9338-023ec6783b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:22.844894 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f57947-6618-4b33-9338-023ec6783b85 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:22.854506 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d954a-eec8-4be3-87bb-0201afdd8654 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:22.855423 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f57947-6618-4b33-9338-023ec6783b85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:07:22.860747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-55f57947-6618-4b33-9338-023ec6783b85 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:07:22.860747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:22.860747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:22.860747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:22.860747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:22.860747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:22.860747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:22.860747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:22.860747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:22.860747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:22.860747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:22.860747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:22.860747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:07:22.860747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:07:22.860747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:07:22.860747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:07:22.861840 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:07:22.861840 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:07:22.861840 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:07:22.861840 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:07:22.861840 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:22.861840 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:22.861840 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:22.861840 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:22.861840 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:22.861840 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:22.861840 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:22.861840 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:22.861840 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:22.861840 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:22.861840 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:22.861840 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:22.862836 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:22.862836 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:22.862836 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:22.862836 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:22.862836 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:07:22.862836 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:07:22.862836 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:07:22.862836 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:07:22.862836 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:07:22.862836 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:07:22.862836 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:07:22.862836 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:22.862836 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 298/571] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Aug 5 14:07:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:22.862836 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f57947-6618-4b33-9338-023ec6783b85 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:22.890096 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67524aae-db45-4170-95d4-cc18bb1954ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:22.890711 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67524aae-db45-4170-95d4-cc18bb1954ce None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:22.890986 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67524aae-db45-4170-95d4-cc18bb1954ce None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:22.891261 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67524aae-db45-4170-95d4-cc18bb1954ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:22.931869 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:22.959099 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67524aae-db45-4170-95d4-cc18bb1954ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:22.960618 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67524aae-db45-4170-95d4-cc18bb1954ce None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:22.965309 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f57947-6618-4b33-9338-023ec6783b85 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:22.971294 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:22.998248 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba6bd1c-272c-4653-a6e4-a83518bf2641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:22.998925 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6bd1c-272c-4653-a6e4-a83518bf2641 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:22.999205 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6bd1c-272c-4653-a6e4-a83518bf2641 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:22.999481 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6bd1c-272c-4653-a6e4-a83518bf2641 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:23.007294 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba6bd1c-272c-4653-a6e4-a83518bf2641 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:23.008728 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba6bd1c-272c-4653-a6e4-a83518bf2641 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:23.019762 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-dba6bd1c-272c-4653-a6e4-a83518bf2641 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 14:07:23.019762 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:23.019762 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:23.019762 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:23.019762 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:23.019762 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:23.019762 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:23.019762 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:23.019762 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:23.019762 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:23.019762 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:23.019762 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:23.019762 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 14:07:23.019762 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 14:07:23.019762 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 14:07:23.019762 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 14:07:23.021453 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:23.021453 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:23.021453 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:07:23.021453 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:23.021453 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:07:23.021453 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:23.021453 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:23.021453 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:23.021453 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:23.021453 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:23.021453 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:23.021453 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:23.021453 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:23.021453 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:23.021453 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:23.021453 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:23.021453 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:23.021453 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:23.022452 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:23.022452 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:23.022452 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 14:07:23.022452 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 14:07:23.022452 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 14:07:23.022452 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 14:07:23.022452 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 14:07:23.022452 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 14:07:23.022452 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 14:07:23.022452 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:23.022452 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 299/572] 10.222.0.26 () {56 vars in 1093 bytes} [Fri Aug 5 14:07:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:23.022452 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f9599-8faa-48f5-9ea7-eccb8b1ac7d0 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:23.022452 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f9599-8faa-48f5-9ea7-eccb8b1ac7d0 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:23.023140 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f9599-8faa-48f5-9ea7-eccb8b1ac7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:23.040683 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-b53f9599-8faa-48f5-9ea7-eccb8b1ac7d0 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:23.041870 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-b53f9599-8faa-48f5-9ea7-eccb8b1ac7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:23.066337 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 274/573] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:07:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 14:07:23.067936 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67524aae-db45-4170-95d4-cc18bb1954ce None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:23.111183 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 300/574] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:07:23.112021 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f9599-8faa-48f5-9ea7-eccb8b1ac7d0 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:23.169933 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:23.177690 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67524aae-db45-4170-95d4-cc18bb1954ce None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:23.199417 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:23.209645 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f9599-8faa-48f5-9ea7-eccb8b1ac7d0 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:23.212068 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb1507-24c1-4890-b3f5-29e7f6e59d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:23.212780 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb1507-24c1-4890-b3f5-29e7f6e59d90 None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:23.213060 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb1507-24c1-4890-b3f5-29e7f6e59d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:23.213336 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb1507-24c1-4890-b3f5-29e7f6e59d90 None admin] PATH_INFO: `/v3/projects/319973e011c54b4ca1f2198c8a5e135b/users/dd1ce2a2b4eb405b999d3fb756795176/roles/ec18a2850fd540a39d4a935348bb4474` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:23.227799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eb1507-24c1-4890-b3f5-29e7f6e59d90 None admin] RBAC: Authorizing `identity:create_grant(project_id=319973e011c54b4ca1f2198c8a5e135b, user_id=dd1ce2a2b4eb405b999d3fb756795176, role_id=ec18a2850fd540a39d4a935348bb4474)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:23.229295 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eb1507-24c1-4890-b3f5-29e7f6e59d90 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:23.243698 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0480c749-2db2-457f-b9d6-ef6a3e39abb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:23.244305 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480c749-2db2-457f-b9d6-ef6a3e39abb2 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:23.244604 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480c749-2db2-457f-b9d6-ef6a3e39abb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:23.244876 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480c749-2db2-457f-b9d6-ef6a3e39abb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:23.245575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0480c749-2db2-457f-b9d6-ef6a3e39abb2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:23.245872 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0480c749-2db2-457f-b9d6-ef6a3e39abb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:23.247462 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0480c749-2db2-457f-b9d6-ef6a3e39abb2 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:23.255181 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 301/575] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Aug 5 14:07:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:23.255938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0480c749-2db2-457f-b9d6-ef6a3e39abb2 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:23.304355 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 275/576] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Aug 5 14:07:23 2022] PUT /identity/v3/projects/319973e011c54b4ca1f2198c8a5e135b/users/dd1ce2a2b4eb405b999d3fb756795176/roles/ec18a2850fd540a39d4a935348bb4474 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:07:23.305232 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb1507-24c1-4890-b3f5-29e7f6e59d90 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:23.371683 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0480c749-2db2-457f-b9d6-ef6a3e39abb2 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:23.398033 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:23.414598 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf0d10-7644-447e-b8a4-a0654ff76617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:23.415227 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf0d10-7644-447e-b8a4-a0654ff76617 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:23.416798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf0d10-7644-447e-b8a4-a0654ff76617 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:23.416798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf0d10-7644-447e-b8a4-a0654ff76617 None admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:23.416798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bf0d10-7644-447e-b8a4-a0654ff76617 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:23.417066 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bf0d10-7644-447e-b8a4-a0654ff76617 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:23.418487 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bf0d10-7644-447e-b8a4-a0654ff76617 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:23.442196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 302/577] 10.222.0.26 () {56 vars in 1133 bytes} [Fri Aug 5 14:07:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:23.442196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf0d10-7644-447e-b8a4-a0654ff76617 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:23.448491 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb1507-24c1-4890-b3f5-29e7f6e59d90 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:23.492048 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125620d1-0afe-4920-9985-e068dd9a78a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:23.493147 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125620d1-0afe-4920-9985-e068dd9a78a6 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:23.493520 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125620d1-0afe-4920-9985-e068dd9a78a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:23.493886 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125620d1-0afe-4920-9985-e068dd9a78a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:23.507344 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125620d1-0afe-4920-9985-e068dd9a78a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:23.508908 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125620d1-0afe-4920-9985-e068dd9a78a6 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:23.526043 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-125620d1-0afe-4920-9985-e068dd9a78a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:23.526043 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:23.526043 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:23.526043 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:23.526043 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:23.526043 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:23.526043 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:23.526043 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:23.526043 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:23.526043 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:23.526043 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:23.526043 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:23.526043 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:23.526043 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:23.526043 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:23.527152 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:23.527152 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:23.527152 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:23.527152 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:23.527152 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:23.527152 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:23.527152 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:23.527152 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:23.527152 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:23.527152 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:23.527152 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:23.527152 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:23.527152 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:23.527152 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:23.527152 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:23.527152 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:23.527152 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:23.528248 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:23.528248 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:23.528248 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:23.528248 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:23.533824 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 276/578] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:23.533824 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125620d1-0afe-4920-9985-e068dd9a78a6 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:23.549544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:23.571253 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf0d10-7644-447e-b8a4-a0654ff76617 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:23.615608 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f18d04f-2323-4d23-9f43-e765e46dfe8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:23.616262 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18d04f-2323-4d23-9f43-e765e46dfe8a None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:23.616572 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18d04f-2323-4d23-9f43-e765e46dfe8a None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:23.616843 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18d04f-2323-4d23-9f43-e765e46dfe8a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:23.631008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f18d04f-2323-4d23-9f43-e765e46dfe8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:07:23.635938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-6f18d04f-2323-4d23-9f43-e765e46dfe8a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:07:23.635938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:23.635938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:23.635938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:23.635938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:23.635938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:23.635938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:23.635938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:23.635938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:23.635938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:23.635938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:23.635938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:23.635938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:07:23.635938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:07:23.635938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:07:23.635938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:07:23.640616 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:07:23.640616 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:07:23.640616 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:07:23.640616 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:07:23.640616 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:23.640616 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:23.640616 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:23.640616 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:23.640616 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:23.640616 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:23.640616 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:23.640616 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:23.640616 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:23.640616 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:23.640616 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:23.640616 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:23.641688 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:23.641688 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:23.641688 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:23.641688 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:23.641688 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:07:23.641688 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:07:23.641688 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:07:23.641688 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:07:23.641688 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:07:23.641688 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:07:23.641688 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:07:23.641688 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:23.641688 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 303/579] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Aug 5 14:07:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:23.641688 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f18d04f-2323-4d23-9f43-e765e46dfe8a None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:23.644653 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125620d1-0afe-4920-9985-e068dd9a78a6 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:23.681558 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:23.683610 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7940c0b-5e48-49d2-bd24-8c21bd32398b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:23.684543 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7940c0b-5e48-49d2-bd24-8c21bd32398b None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:23.684968 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7940c0b-5e48-49d2-bd24-8c21bd32398b None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:23.685399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7940c0b-5e48-49d2-bd24-8c21bd32398b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:23.698295 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7940c0b-5e48-49d2-bd24-8c21bd32398b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:23.699745 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7940c0b-5e48-49d2-bd24-8c21bd32398b None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:23.716143 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-c7940c0b-5e48-49d2-bd24-8c21bd32398b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:23.716143 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:23.716143 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:23.716143 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:23.716143 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:23.716143 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:23.716143 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:23.716143 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:23.716143 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:23.716143 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:23.716143 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:23.716143 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:23.716143 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:23.716143 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:23.716143 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:23.717312 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:23.717312 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:23.717312 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:23.717312 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:23.717312 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:23.717312 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:23.717312 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:23.717312 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:23.717312 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:23.717312 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:23.717312 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:23.717312 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:23.717312 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:23.717312 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:23.717312 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:23.717312 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:23.717312 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:23.723142 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:23.723142 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:23.723142 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:23.723142 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:23.723142 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 277/580] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:23.723142 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7940c0b-5e48-49d2-bd24-8c21bd32398b None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:23.750511 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f18d04f-2323-4d23-9f43-e765e46dfe8a None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:23.786599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:23.787387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605a36b-d20a-4cc7-83a4-23302b6b9933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:23.788247 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605a36b-d20a-4cc7-83a4-23302b6b9933 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:23.788624 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605a36b-d20a-4cc7-83a4-23302b6b9933 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:23.788964 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605a36b-d20a-4cc7-83a4-23302b6b9933 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:23.799464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5605a36b-d20a-4cc7-83a4-23302b6b9933 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:23.801197 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5605a36b-d20a-4cc7-83a4-23302b6b9933 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:23.816307 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-5605a36b-d20a-4cc7-83a4-23302b6b9933 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:23.816307 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:23.816307 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:23.816307 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:23.816307 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:23.816307 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:23.816307 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:23.816307 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:23.816307 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:23.816307 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:23.816307 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:23.816307 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:23.816307 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:23.816307 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:23.816307 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:23.817956 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:23.817956 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:23.817956 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:23.817956 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:23.817956 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:23.817956 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:23.817956 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:23.817956 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:23.817956 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:23.817956 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:23.817956 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:23.817956 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:23.817956 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:23.817956 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:23.817956 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:23.817956 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:23.817956 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:23.819056 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:23.819056 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:23.819056 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:23.819056 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:23.824342 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 304/581] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:23.824342 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605a36b-d20a-4cc7-83a4-23302b6b9933 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:23.832012 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7940c0b-5e48-49d2-bd24-8c21bd32398b None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:23.869340 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72cc1e5-0968-4d62-a49c-a16d0eb4c522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:23.869963 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72cc1e5-0968-4d62-a49c-a16d0eb4c522 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:23.870243 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72cc1e5-0968-4d62-a49c-a16d0eb4c522 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:23.870518 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72cc1e5-0968-4d62-a49c-a16d0eb4c522 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:23.871209 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72cc1e5-0968-4d62-a49c-a16d0eb4c522 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:23.871773 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72cc1e5-0968-4d62-a49c-a16d0eb4c522 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:23.873170 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72cc1e5-0968-4d62-a49c-a16d0eb4c522 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:23.885711 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 278/582] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:23.886461 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72cc1e5-0968-4d62-a49c-a16d0eb4c522 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:23.930514 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:23.940329 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605a36b-d20a-4cc7-83a4-23302b6b9933 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:23.975004 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f001e-1640-479a-a2e2-499a295e7af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:23.975625 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f001e-1640-479a-a2e2-499a295e7af6 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:23.975906 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f001e-1640-479a-a2e2-499a295e7af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:23.976180 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f001e-1640-479a-a2e2-499a295e7af6 None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:23.976918 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5f001e-1640-479a-a2e2-499a295e7af6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:23.977226 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5f001e-1640-479a-a2e2-499a295e7af6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:23.978628 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5f001e-1640-479a-a2e2-499a295e7af6 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:23.989746 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 305/583] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Aug 5 14:07:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:23.990946 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f001e-1640-479a-a2e2-499a295e7af6 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:23.999151 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72cc1e5-0968-4d62-a49c-a16d0eb4c522 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:24.024290 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:24.036800 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e283de-9968-46d5-9bb2-fb20b73a3166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:24.037429 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e283de-9968-46d5-9bb2-fb20b73a3166 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:24.037703 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e283de-9968-46d5-9bb2-fb20b73a3166 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:24.038018 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e283de-9968-46d5-9bb2-fb20b73a3166 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:24.039164 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e283de-9968-46d5-9bb2-fb20b73a3166 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:24.039473 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e283de-9968-46d5-9bb2-fb20b73a3166 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:24.041514 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e283de-9968-46d5-9bb2-fb20b73a3166 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:24.059550 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 279/584] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:24.060870 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e283de-9968-46d5-9bb2-fb20b73a3166 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:24.105148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f001e-1640-479a-a2e2-499a295e7af6 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:24.150383 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4fe346-d9e9-4e6b-b14c-6abbfef1723e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:24.151006 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fe346-d9e9-4e6b-b14c-6abbfef1723e None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:24.151351 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fe346-d9e9-4e6b-b14c-6abbfef1723e None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:24.151690 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fe346-d9e9-4e6b-b14c-6abbfef1723e None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:24.152528 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4fe346-d9e9-4e6b-b14c-6abbfef1723e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:24.152885 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4fe346-d9e9-4e6b-b14c-6abbfef1723e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:24.154313 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4fe346-d9e9-4e6b-b14c-6abbfef1723e None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:24.165121 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e283de-9968-46d5-9bb2-fb20b73a3166 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:24.173540 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 306/585] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:24.173540 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4fe346-d9e9-4e6b-b14c-6abbfef1723e None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:24.188117 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:24.204823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e16ab-43f1-47a5-9ad1-04165e61fabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:24.205462 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e16ab-43f1-47a5-9ad1-04165e61fabb None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:24.205740 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e16ab-43f1-47a5-9ad1-04165e61fabb None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:24.206118 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e16ab-43f1-47a5-9ad1-04165e61fabb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:24.213244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e16ab-43f1-47a5-9ad1-04165e61fabb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:24.214692 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e16ab-43f1-47a5-9ad1-04165e61fabb None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:24.226902 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-198e16ab-43f1-47a5-9ad1-04165e61fabb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:24.226902 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:24.226902 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:24.226902 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:24.226902 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:24.226902 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:24.226902 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:24.226902 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:24.226902 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:24.226902 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:24.226902 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:24.226902 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:24.226902 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 14:07:24.226902 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 14:07:24.226902 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 14:07:24.227990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 14:07:24.227990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:24.227990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:24.227990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:24.227990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:24.227990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:24.227990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:24.227990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:24.227990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:24.227990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:24.227990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:24.227990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:24.227990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:24.227990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:24.227990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:24.227990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:24.227990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:24.228941 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 14:07:24.228941 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 14:07:24.228941 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:07:24.228941 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:07:24.228941 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:07:24.228941 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:07:24.228941 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:24.228941 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:24.228941 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 280/586] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:07:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:24.228941 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e16ab-43f1-47a5-9ad1-04165e61fabb None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:24.279909 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4fe346-d9e9-4e6b-b14c-6abbfef1723e None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:24.288116 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:24.315347 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207815d7-6c6b-4632-8550-793f1b19f632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:24.315984 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207815d7-6c6b-4632-8550-793f1b19f632 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:24.316260 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207815d7-6c6b-4632-8550-793f1b19f632 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:24.317118 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207815d7-6c6b-4632-8550-793f1b19f632 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:24.327290 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207815d7-6c6b-4632-8550-793f1b19f632 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:24.328745 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207815d7-6c6b-4632-8550-793f1b19f632 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:24.343456 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-207815d7-6c6b-4632-8550-793f1b19f632 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:24.343456 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:24.343456 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:24.343456 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:24.343456 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:24.343456 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:24.343456 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:24.343456 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:24.343456 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:24.343456 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:24.343456 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:24.343456 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:24.343456 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:24.343456 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:24.343456 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:24.344575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:24.344575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:24.344575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:24.344575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:24.344575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:24.344575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:24.344575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:24.344575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:24.344575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:24.344575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:24.344575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:24.344575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:24.344575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:24.344575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:24.344575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:24.344575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:24.344575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:24.345676 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:24.345676 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:24.345676 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:24.345676 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:24.349805 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 307/587] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:24.350579 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207815d7-6c6b-4632-8550-793f1b19f632 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:24.353933 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e16ab-43f1-47a5-9ad1-04165e61fabb None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:24.390920 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe908a-8297-495f-b410-9476779332ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:24.391717 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe908a-8297-495f-b410-9476779332ed None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:24.392112 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe908a-8297-495f-b410-9476779332ed None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:24.392508 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe908a-8297-495f-b410-9476779332ed None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:24.400619 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fe908a-8297-495f-b410-9476779332ed None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:24.402117 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fe908a-8297-495f-b410-9476779332ed None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:24.416651 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-b9fe908a-8297-495f-b410-9476779332ed None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 14:07:24.416651 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:24.416651 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:24.416651 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:24.416651 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:24.416651 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:24.416651 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:24.416651 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:24.416651 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:24.416651 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:24.416651 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:24.416651 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:24.416651 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 14:07:24.416651 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 14:07:24.416651 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 14:07:24.416651 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 14:07:24.417678 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:24.417678 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:24.417678 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:07:24.417678 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:24.417678 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:07:24.417678 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:24.417678 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:24.417678 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:24.417678 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:24.417678 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:24.417678 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:24.417678 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:24.417678 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:24.417678 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:24.417678 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:24.417678 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:24.417678 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:24.417678 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:24.418620 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:24.418620 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:24.418620 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 14:07:24.418620 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 14:07:24.418620 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 14:07:24.418620 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 14:07:24.418620 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 14:07:24.418620 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 14:07:24.418620 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 14:07:24.418620 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:24.418620 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 281/588] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:07:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:24.421638 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe908a-8297-495f-b410-9476779332ed None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:24.428003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:24.467934 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207815d7-6c6b-4632-8550-793f1b19f632 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:24.508144 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd27a0-3377-48bf-a440-31316e8273f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:24.509415 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd27a0-3377-48bf-a440-31316e8273f6 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:24.510094 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd27a0-3377-48bf-a440-31316e8273f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:24.510547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd27a0-3377-48bf-a440-31316e8273f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:24.520655 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fd27a0-3377-48bf-a440-31316e8273f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:24.523942 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fd27a0-3377-48bf-a440-31316e8273f6 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:24.532511 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe908a-8297-495f-b410-9476779332ed None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:24.533303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-d0fd27a0-3377-48bf-a440-31316e8273f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:24.533303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:24.533303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:24.533303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:24.533303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:24.533303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:24.533303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:24.533303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:24.533303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:24.533303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:24.533303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:24.533303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:24.533303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 14:07:24.533303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 14:07:24.533303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 14:07:24.534355 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 14:07:24.534355 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:24.534355 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:24.534355 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:24.534355 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:24.534355 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:24.534355 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:24.534355 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:24.534355 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:24.534355 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:24.534355 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:24.534355 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:24.534355 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:24.534355 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:24.534355 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:24.534355 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:24.534355 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:24.536001 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 14:07:24.536001 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 14:07:24.536001 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:07:24.536001 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:07:24.536001 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:07:24.536001 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:07:24.536001 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:24.536001 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:24.538846 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:24.540068 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 308/589] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:07:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:24.540880 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd27a0-3377-48bf-a440-31316e8273f6 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:24.571922 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366a152-e0cc-4dbb-b1c3-d6752a2e621b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:24.572583 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366a152-e0cc-4dbb-b1c3-d6752a2e621b None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:24.572864 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366a152-e0cc-4dbb-b1c3-d6752a2e621b None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:24.573137 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366a152-e0cc-4dbb-b1c3-d6752a2e621b None admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:24.573883 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b366a152-e0cc-4dbb-b1c3-d6752a2e621b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:24.574193 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b366a152-e0cc-4dbb-b1c3-d6752a2e621b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:24.575658 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b366a152-e0cc-4dbb-b1c3-d6752a2e621b None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:24.587801 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 282/590] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Aug 5 14:07:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:24.588624 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366a152-e0cc-4dbb-b1c3-d6752a2e621b None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:24.672788 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd27a0-3377-48bf-a440-31316e8273f6 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:24.700292 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:24.714127 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8013a2-5416-4968-85ba-8a5b76b7d8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:24.714749 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8013a2-5416-4968-85ba-8a5b76b7d8f5 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:24.715039 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8013a2-5416-4968-85ba-8a5b76b7d8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:24.715315 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8013a2-5416-4968-85ba-8a5b76b7d8f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:24.715975 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8013a2-5416-4968-85ba-8a5b76b7d8f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:24.716277 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8013a2-5416-4968-85ba-8a5b76b7d8f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:24.717124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366a152-e0cc-4dbb-b1c3-d6752a2e621b None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:24.717674 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8013a2-5416-4968-85ba-8a5b76b7d8f5 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:24.729733 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 309/591] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:24.730612 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8013a2-5416-4968-85ba-8a5b76b7d8f5 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:24.759648 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9c59ea-2e43-4bdc-a42a-761f9ca9e280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:24.760363 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c59ea-2e43-4bdc-a42a-761f9ca9e280 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:24.760679 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c59ea-2e43-4bdc-a42a-761f9ca9e280 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:24.760955 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c59ea-2e43-4bdc-a42a-761f9ca9e280 None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:24.761749 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9c59ea-2e43-4bdc-a42a-761f9ca9e280 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:24.762059 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9c59ea-2e43-4bdc-a42a-761f9ca9e280 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:24.763497 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9c59ea-2e43-4bdc-a42a-761f9ca9e280 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:24.782816 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 283/592] 10.222.0.26 () {56 vars in 1137 bytes} [Fri Aug 5 14:07:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:24.783564 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9c59ea-2e43-4bdc-a42a-761f9ca9e280 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:24.808519 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:24.841930 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8013a2-5416-4968-85ba-8a5b76b7d8f5 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:24.878798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247ed4a-4a81-4df4-be31-383aece2bdd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:24.879995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247ed4a-4a81-4df4-be31-383aece2bdd9 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:24.880269 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247ed4a-4a81-4df4-be31-383aece2bdd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:24.880567 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247ed4a-4a81-4df4-be31-383aece2bdd9 None admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:24.881311 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5247ed4a-4a81-4df4-be31-383aece2bdd9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:24.881703 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5247ed4a-4a81-4df4-be31-383aece2bdd9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:24.883044 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5247ed4a-4a81-4df4-be31-383aece2bdd9 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:24.893053 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9c59ea-2e43-4bdc-a42a-761f9ca9e280 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:24.900402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 310/593] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Aug 5 14:07:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:24.900402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247ed4a-4a81-4df4-be31-383aece2bdd9 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:24.936179 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf4d98-059f-4f42-81ac-87179ef58816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:24.936830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf4d98-059f-4f42-81ac-87179ef58816 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:24.937183 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf4d98-059f-4f42-81ac-87179ef58816 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:24.937465 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf4d98-059f-4f42-81ac-87179ef58816 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:24.938254 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacf4d98-059f-4f42-81ac-87179ef58816 None admin] RBAC: Adding query filter params (role.id=ec18a2850fd540a39d4a935348bb4474, scope.project.id=319973e011c54b4ca1f2198c8a5e135b, user.id=dddd7eea67f34f0cbb5f6a1cd85f1fc5) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:24.938560 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacf4d98-059f-4f42-81ac-87179ef58816 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:24.940001 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacf4d98-059f-4f42-81ac-87179ef58816 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:24.945913 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 284/594] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:07:24 2022] GET /identity/v3/role_assignments?user.id=dddd7eea67f34f0cbb5f6a1cd85f1fc5&scope.project.id=319973e011c54b4ca1f2198c8a5e135b&role.id=ec18a2850fd540a39d4a935348bb4474 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:24.949505 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:24.958023 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf4d98-059f-4f42-81ac-87179ef58816 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:25.021593 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247ed4a-4a81-4df4-be31-383aece2bdd9 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:25.064944 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006dce7a-e620-4cbc-907f-1ae22a1732eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:25.065896 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006dce7a-e620-4cbc-907f-1ae22a1732eb None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:25.066330 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006dce7a-e620-4cbc-907f-1ae22a1732eb None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:25.067409 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006dce7a-e620-4cbc-907f-1ae22a1732eb None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:25.068973 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:25.077730 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006dce7a-e620-4cbc-907f-1ae22a1732eb None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:25.079533 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006dce7a-e620-4cbc-907f-1ae22a1732eb None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:25.086443 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf4d98-059f-4f42-81ac-87179ef58816 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:25.093905 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-006dce7a-e620-4cbc-907f-1ae22a1732eb None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 14:07:25.093905 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:25.093905 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:25.093905 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:25.093905 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:25.093905 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:25.093905 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:25.093905 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:25.093905 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:25.093905 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:25.093905 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:25.093905 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:25.093905 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 14:07:25.093905 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 14:07:25.093905 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 14:07:25.093905 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 14:07:25.095008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:25.095008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:25.095008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:07:25.095008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:25.095008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:07:25.095008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:25.095008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:25.095008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:25.095008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:25.095008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:25.095008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:25.095008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:25.095008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:25.095008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:25.095008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:25.095008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:25.095008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:25.095008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:25.095928 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:25.095928 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:25.095928 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 14:07:25.095928 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 14:07:25.095928 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 14:07:25.095928 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 14:07:25.095928 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 14:07:25.095928 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 14:07:25.095928 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 14:07:25.095928 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:25.099765 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 311/595] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Aug 5 14:07:24 2022] GET /identity/v3/users/neutron => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:25.100860 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006dce7a-e620-4cbc-907f-1ae22a1732eb None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:25.131009 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8005bfe1-bb70-40fe-8c67-facfd7a4223a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:25.131768 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005bfe1-bb70-40fe-8c67-facfd7a4223a None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:25.132361 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005bfe1-bb70-40fe-8c67-facfd7a4223a None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:25.132361 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005bfe1-bb70-40fe-8c67-facfd7a4223a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:25.143374 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8005bfe1-bb70-40fe-8c67-facfd7a4223a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:25.144450 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8005bfe1-bb70-40fe-8c67-facfd7a4223a None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:25.161278 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-8005bfe1-bb70-40fe-8c67-facfd7a4223a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:25.161278 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:25.161278 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:25.161278 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:25.161278 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:25.161278 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:25.161278 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:25.161278 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:25.161278 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:25.161278 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:25.161278 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:25.161278 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:25.161278 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:25.161278 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:25.161278 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:25.162205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:25.162205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:25.162205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:25.162205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:25.162205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:25.162205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:25.162205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:25.162205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:25.162205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:25.162205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:25.162205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:25.162205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:25.162205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:25.162205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:25.162205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:25.162205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:25.162205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:25.163095 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:25.163095 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:25.163095 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:25.163095 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:25.167246 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 285/596] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:25.168154 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8005bfe1-bb70-40fe-8c67-facfd7a4223a None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:25.219487 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006dce7a-e620-4cbc-907f-1ae22a1732eb None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:25.222906 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:25.255610 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a58e5b4-04f0-4854-9ae5-1581bc6470a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:25.256613 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58e5b4-04f0-4854-9ae5-1581bc6470a2 None admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:25.257094 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58e5b4-04f0-4854-9ae5-1581bc6470a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:25.257552 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58e5b4-04f0-4854-9ae5-1581bc6470a2 None admin] PATH_INFO: `/v3/projects/319973e011c54b4ca1f2198c8a5e135b/users/5ec4940c99fc4fa8a5bc7cdabc4558fb/roles/ec18a2850fd540a39d4a935348bb4474` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:25.273571 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a58e5b4-04f0-4854-9ae5-1581bc6470a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=319973e011c54b4ca1f2198c8a5e135b, user_id=5ec4940c99fc4fa8a5bc7cdabc4558fb, role_id=ec18a2850fd540a39d4a935348bb4474)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:25.275144 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a58e5b4-04f0-4854-9ae5-1581bc6470a2 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:25.301657 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8005bfe1-bb70-40fe-8c67-facfd7a4223a None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:25.337789 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c7918d-6ec0-4a6d-8291-3736a8a7a70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:25.338404 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c7918d-6ec0-4a6d-8291-3736a8a7a70b None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:25.338681 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c7918d-6ec0-4a6d-8291-3736a8a7a70b None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:25.338946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c7918d-6ec0-4a6d-8291-3736a8a7a70b None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:25.344293 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c7918d-6ec0-4a6d-8291-3736a8a7a70b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:25.344293 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c7918d-6ec0-4a6d-8291-3736a8a7a70b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:25.344293 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c7918d-6ec0-4a6d-8291-3736a8a7a70b None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:25.350883 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:25.359901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 286/597] 10.222.0.26 () {56 vars in 1139 bytes} [Fri Aug 5 14:07:25 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:25.360792 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c7918d-6ec0-4a6d-8291-3736a8a7a70b None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:25.385417 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 312/598] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Aug 5 14:07:25 2022] PUT /identity/v3/projects/319973e011c54b4ca1f2198c8a5e135b/users/5ec4940c99fc4fa8a5bc7cdabc4558fb/roles/ec18a2850fd540a39d4a935348bb4474 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:07:25.386999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e8af9-cf58-4130-b6fa-c5c676e27247 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:25.387515 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e8af9-cf58-4130-b6fa-c5c676e27247 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:25.388012 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e8af9-cf58-4130-b6fa-c5c676e27247 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:25.390021 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 313/599] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:25.391157 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e8af9-cf58-4130-b6fa-c5c676e27247 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:25.503049 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c7918d-6ec0-4a6d-8291-3736a8a7a70b None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:25.516773 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e8af9-cf58-4130-b6fa-c5c676e27247 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:25.549932 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc81729-00ac-4398-b5b6-dc3f7832ac16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:25.553377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc81729-00ac-4398-b5b6-dc3f7832ac16 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:25.553377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc81729-00ac-4398-b5b6-dc3f7832ac16 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:25.553377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc81729-00ac-4398-b5b6-dc3f7832ac16 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:25.553802 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc81729-00ac-4398-b5b6-dc3f7832ac16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:25.554108 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc81729-00ac-4398-b5b6-dc3f7832ac16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:25.556446 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc81729-00ac-4398-b5b6-dc3f7832ac16 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:25.562765 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70524166-9272-4632-97ea-9517989ba8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:25.563400 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70524166-9272-4632-97ea-9517989ba8ac None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:25.563728 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70524166-9272-4632-97ea-9517989ba8ac None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:25.564077 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70524166-9272-4632-97ea-9517989ba8ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:25.569356 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 287/600] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:25.570719 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70078d49-a9f6-49e8-a1a4-6a03d31c6ae6 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:25.570950 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70078d49-a9f6-49e8-a1a4-6a03d31c6ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:25.571225 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70078d49-a9f6-49e8-a1a4-6a03d31c6ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:25.575849 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70524166-9272-4632-97ea-9517989ba8ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:25.578860 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70524166-9272-4632-97ea-9517989ba8ac None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:25.591621 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-70078d49-a9f6-49e8-a1a4-6a03d31c6ae6 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:25.592862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-70078d49-a9f6-49e8-a1a4-6a03d31c6ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:25.593890 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-70524166-9272-4632-97ea-9517989ba8ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:25.593890 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:25.593890 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:25.593890 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:25.593890 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:25.593890 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:25.593890 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:25.593890 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:25.593890 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:25.593890 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:25.593890 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:25.593890 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:25.593890 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:25.593890 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:25.593890 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:25.594750 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:25.594750 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:25.594750 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:25.594750 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:25.594750 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:25.594750 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:25.594750 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:25.594750 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:25.594750 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:25.594750 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:25.594750 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:25.594750 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:25.594750 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:25.594750 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:25.594750 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:25.594750 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:25.594750 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:25.595632 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:25.595632 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:25.595632 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:25.595632 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:25.595632 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 314/601] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:25.596676 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70524166-9272-4632-97ea-9517989ba8ac None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:25.609597 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:25.664827 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:25.687014 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 288/602] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:07:25.688738 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70078d49-a9f6-49e8-a1a4-6a03d31c6ae6 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:25.729339 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70524166-9272-4632-97ea-9517989ba8ac None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:25.768631 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb397ae-80dd-4964-855a-e58c0c4294ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:25.769894 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb397ae-80dd-4964-855a-e58c0c4294ee None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:25.770469 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb397ae-80dd-4964-855a-e58c0c4294ee None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:25.770967 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb397ae-80dd-4964-855a-e58c0c4294ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:25.778671 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb397ae-80dd-4964-855a-e58c0c4294ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:25.780381 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb397ae-80dd-4964-855a-e58c0c4294ee None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:25.792911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-feb397ae-80dd-4964-855a-e58c0c4294ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:25.792911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:25.792911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:25.792911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:25.792911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:25.792911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:25.792911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:25.792911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:25.792911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:25.792911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:25.792911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:25.792911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:25.792911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 14:07:25.792911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 14:07:25.792911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 14:07:25.793818 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 14:07:25.793818 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:25.793818 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:25.793818 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:25.793818 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:25.793818 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:25.793818 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:25.793818 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:25.793818 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:25.793818 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:25.793818 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:25.793818 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:25.793818 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:25.793818 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:25.793818 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:25.793818 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:25.793818 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:25.794691 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 14:07:25.794691 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 14:07:25.794691 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:07:25.794691 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:07:25.794691 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:07:25.794691 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:07:25.794691 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:25.794691 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:25.797225 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 315/603] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:07:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:25.803236 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb397ae-80dd-4964-855a-e58c0c4294ee None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:25.809787 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70078d49-a9f6-49e8-a1a4-6a03d31c6ae6 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:25.848207 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b54b82-694d-4391-8b28-796c1de499d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:25.848894 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b54b82-694d-4391-8b28-796c1de499d6 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:25.849169 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b54b82-694d-4391-8b28-796c1de499d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:25.849436 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b54b82-694d-4391-8b28-796c1de499d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:25.850175 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b54b82-694d-4391-8b28-796c1de499d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:25.850545 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b54b82-694d-4391-8b28-796c1de499d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:25.852010 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b54b82-694d-4391-8b28-796c1de499d6 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:25.865208 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 289/604] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:25.866033 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b54b82-694d-4391-8b28-796c1de499d6 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:25.879835 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:25.918814 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb397ae-80dd-4964-855a-e58c0c4294ee None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:25.922887 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:25.957935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d9ab89-1779-4e6c-9fc8-a9fee1eb3e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:25.958988 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9ab89-1779-4e6c-9fc8-a9fee1eb3e17 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:25.959472 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9ab89-1779-4e6c-9fc8-a9fee1eb3e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:25.959970 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9ab89-1779-4e6c-9fc8-a9fee1eb3e17 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:25.971725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d9ab89-1779-4e6c-9fc8-a9fee1eb3e17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:07:25.978656 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-a5d9ab89-1779-4e6c-9fc8-a9fee1eb3e17 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:07:25.978656 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:25.978656 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:25.978656 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:25.978656 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:25.978656 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:25.978656 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:25.978656 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:25.978656 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:25.978656 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:25.978656 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:25.978656 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:25.978656 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:07:25.978656 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:07:25.978656 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:07:25.978656 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:07:25.979592 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:07:25.979592 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:07:25.979592 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:07:25.979592 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:07:25.979592 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:25.979592 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:25.979592 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:25.979592 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:25.979592 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:25.979592 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:25.979592 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:25.979592 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:25.979592 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:25.979592 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:25.979592 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:25.979592 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:25.980457 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:25.980457 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:25.980457 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:25.980457 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:25.980457 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:07:25.980457 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:07:25.980457 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:07:25.980457 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:07:25.980457 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:07:25.980457 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:07:25.980457 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:07:25.980457 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:25.981124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b54b82-694d-4391-8b28-796c1de499d6 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:25.983355 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 316/605] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Aug 5 14:07:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:25.984424 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d9ab89-1779-4e6c-9fc8-a9fee1eb3e17 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:26.023623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d2c64-98ec-4346-8a7a-d6d1786d63a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:26.024170 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d2c64-98ec-4346-8a7a-d6d1786d63a7 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:26.024478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d2c64-98ec-4346-8a7a-d6d1786d63a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:26.024756 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d2c64-98ec-4346-8a7a-d6d1786d63a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:26.025561 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86d2c64-98ec-4346-8a7a-d6d1786d63a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:26.025871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86d2c64-98ec-4346-8a7a-d6d1786d63a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:26.027283 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86d2c64-98ec-4346-8a7a-d6d1786d63a7 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:26.045751 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 290/606] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Aug 5 14:07:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:26.045751 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d2c64-98ec-4346-8a7a-d6d1786d63a7 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:26.106664 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d9ab89-1779-4e6c-9fc8-a9fee1eb3e17 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:26.141339 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:26.152894 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ad22e-04d1-4ff2-98ef-465e1127ebc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:26.157497 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ad22e-04d1-4ff2-98ef-465e1127ebc7 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:26.157497 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ad22e-04d1-4ff2-98ef-465e1127ebc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:26.157497 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ad22e-04d1-4ff2-98ef-465e1127ebc7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:26.163363 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ad22e-04d1-4ff2-98ef-465e1127ebc7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:26.165778 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ad22e-04d1-4ff2-98ef-465e1127ebc7 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:26.174536 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d2c64-98ec-4346-8a7a-d6d1786d63a7 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:26.181013 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-949ad22e-04d1-4ff2-98ef-465e1127ebc7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:26.181013 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:26.181013 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:26.181013 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:26.181013 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:26.181013 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:26.181013 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:26.181013 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:26.181013 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:26.181013 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:26.181013 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:26.181013 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:26.181013 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 14:07:26.181013 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 14:07:26.181013 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 14:07:26.182119 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 14:07:26.182119 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:26.182119 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:26.182119 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:26.182119 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:26.182119 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:26.182119 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:26.182119 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:26.182119 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:26.182119 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:26.182119 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:26.182119 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:26.182119 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:26.182119 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:26.182119 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:26.182119 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:26.182119 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:26.183184 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 14:07:26.183184 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 14:07:26.183184 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:07:26.183184 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:07:26.183184 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:07:26.183184 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:07:26.183184 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:26.183184 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:26.190116 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:26.191463 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 317/607] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:07:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:26.194232 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ad22e-04d1-4ff2-98ef-465e1127ebc7 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:26.222427 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e6640d-43aa-4c1c-a6f9-526206c0c986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:26.225285 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6640d-43aa-4c1c-a6f9-526206c0c986 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:26.225682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6640d-43aa-4c1c-a6f9-526206c0c986 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:26.226060 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6640d-43aa-4c1c-a6f9-526206c0c986 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:26.227518 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e6640d-43aa-4c1c-a6f9-526206c0c986 None admin] RBAC: Adding query filter params (name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:26.227929 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e6640d-43aa-4c1c-a6f9-526206c0c986 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:26.229990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e6640d-43aa-4c1c-a6f9-526206c0c986 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:26.244465 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 291/608] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Aug 5 14:07:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:26.244465 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e6640d-43aa-4c1c-a6f9-526206c0c986 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:26.319580 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ad22e-04d1-4ff2-98ef-465e1127ebc7 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:26.361332 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e26636-4ce5-4668-82bd-68154c423f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:26.362661 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e26636-4ce5-4668-82bd-68154c423f6c None admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:26.363285 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e26636-4ce5-4668-82bd-68154c423f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:26.363735 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e26636-4ce5-4668-82bd-68154c423f6c None admin] PATH_INFO: `/v3/projects/319973e011c54b4ca1f2198c8a5e135b/users/88d6a1bae9834e2e91d4ab3a4c00959b/roles/ec18a2850fd540a39d4a935348bb4474` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:26.375829 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e6640d-43aa-4c1c-a6f9-526206c0c986 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:26.382068 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e26636-4ce5-4668-82bd-68154c423f6c None admin] RBAC: Authorizing `identity:create_grant(project_id=319973e011c54b4ca1f2198c8a5e135b, user_id=88d6a1bae9834e2e91d4ab3a4c00959b, role_id=ec18a2850fd540a39d4a935348bb4474)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:26.383661 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e26636-4ce5-4668-82bd-68154c423f6c None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:26.413210 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f19fc-4215-4a6a-b9c1-f7160b74d008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:26.413979 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f19fc-4215-4a6a-b9c1-f7160b74d008 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:26.414368 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f19fc-4215-4a6a-b9c1-f7160b74d008 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:26.414732 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f19fc-4215-4a6a-b9c1-f7160b74d008 None admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:26.415588 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9f19fc-4215-4a6a-b9c1-f7160b74d008 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:26.416011 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9f19fc-4215-4a6a-b9c1-f7160b74d008 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:26.417683 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9f19fc-4215-4a6a-b9c1-f7160b74d008 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:26.435579 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 292/609] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Aug 5 14:07:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:26.435579 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f19fc-4215-4a6a-b9c1-f7160b74d008 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:26.462526 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:26.496588 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 318/610] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Aug 5 14:07:26 2022] PUT /identity/v3/projects/319973e011c54b4ca1f2198c8a5e135b/users/88d6a1bae9834e2e91d4ab3a4c00959b/roles/ec18a2850fd540a39d4a935348bb4474 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:07:26.497714 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e26636-4ce5-4668-82bd-68154c423f6c None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:26.557619 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f19fc-4215-4a6a-b9c1-f7160b74d008 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:26.587840 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:26.601389 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327cdae1-6f17-46ba-8286-0764a07c5673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:26.602050 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327cdae1-6f17-46ba-8286-0764a07c5673 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:26.602322 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327cdae1-6f17-46ba-8286-0764a07c5673 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:26.602587 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327cdae1-6f17-46ba-8286-0764a07c5673 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:26.624008 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327cdae1-6f17-46ba-8286-0764a07c5673 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:26.625759 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327cdae1-6f17-46ba-8286-0764a07c5673 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:26.657579 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-327cdae1-6f17-46ba-8286-0764a07c5673 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:26.657579 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:26.657579 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:26.657579 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:26.657579 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:26.657579 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:26.657579 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:26.657579 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:26.657579 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:26.657579 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:26.657579 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:26.657579 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:26.657579 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:26.657579 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:26.657579 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:26.658811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:26.658811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:26.658811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:26.658811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:26.658811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:26.658811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:26.658811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:26.658811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:26.658811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:26.658811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:26.658811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:26.658811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:26.658811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:26.658811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:26.658811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:26.658811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:26.658811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:26.659829 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:26.659829 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:26.659829 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:26.659829 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:26.659829 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 293/611] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:26.666736 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327cdae1-6f17-46ba-8286-0764a07c5673 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:26.741695 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e26636-4ce5-4668-82bd-68154c423f6c None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:26.769320 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:26.808870 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327cdae1-6f17-46ba-8286-0764a07c5673 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:26.825384 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e16c1f-c2d1-4cfb-bc3c-27e090e130f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:26.826062 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e16c1f-c2d1-4cfb-bc3c-27e090e130f2 None admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:26.826334 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e16c1f-c2d1-4cfb-bc3c-27e090e130f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:26.826603 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e16c1f-c2d1-4cfb-bc3c-27e090e130f2 None admin] PATH_INFO: `/v3/projects/319973e011c54b4ca1f2198c8a5e135b/users/695e37560f8f495e987d1bd17acc2680/roles/ec18a2850fd540a39d4a935348bb4474` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:26.854044 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e16c1f-c2d1-4cfb-bc3c-27e090e130f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=319973e011c54b4ca1f2198c8a5e135b, user_id=695e37560f8f495e987d1bd17acc2680, role_id=ec18a2850fd540a39d4a935348bb4474)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:26.855781 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e16c1f-c2d1-4cfb-bc3c-27e090e130f2 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:26.865075 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9865ddba-61bb-4957-88fd-7b438f8e2f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:26.865699 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865ddba-61bb-4957-88fd-7b438f8e2f73 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:26.865972 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865ddba-61bb-4957-88fd-7b438f8e2f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:26.866240 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865ddba-61bb-4957-88fd-7b438f8e2f73 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:26.866948 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9865ddba-61bb-4957-88fd-7b438f8e2f73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:26.867227 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9865ddba-61bb-4957-88fd-7b438f8e2f73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:26.872471 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9865ddba-61bb-4957-88fd-7b438f8e2f73 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:26.881446 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 294/612] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:26.888551 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9865ddba-61bb-4957-88fd-7b438f8e2f73 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:26.927713 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 319/613] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Aug 5 14:07:26 2022] PUT /identity/v3/projects/319973e011c54b4ca1f2198c8a5e135b/users/695e37560f8f495e987d1bd17acc2680/roles/ec18a2850fd540a39d4a935348bb4474 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:07:27.034199 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9865ddba-61bb-4957-88fd-7b438f8e2f73 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:27.086663 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec2be3-ed2f-4f8a-8e07-758b719acbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:27.087328 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec2be3-ed2f-4f8a-8e07-758b719acbff None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:27.087609 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec2be3-ed2f-4f8a-8e07-758b719acbff None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:27.087890 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec2be3-ed2f-4f8a-8e07-758b719acbff None admin] PATH_INFO: `/v3/users/glance` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:27.099341 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ec2be3-ed2f-4f8a-8e07-758b719acbff None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:27.100830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ec2be3-ed2f-4f8a-8e07-758b719acbff None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:27.107287 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30045039-2b20-4dfe-98c8-faa37d6cbd67 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:27.107492 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30045039-2b20-4dfe-98c8-faa37d6cbd67 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:27.107769 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30045039-2b20-4dfe-98c8-faa37d6cbd67 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:27.109431 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 320/614] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:07:27.111520 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:27.116202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c46d4-22c1-4b45-89af-55cda4aa252a None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:27.116514 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c46d4-22c1-4b45-89af-55cda4aa252a None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:27.116792 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c46d4-22c1-4b45-89af-55cda4aa252a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:27.118371 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-38ec2be3-ed2f-4f8a-8e07-758b719acbff None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 05 14:07:27.118371 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:27.118371 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:27.118371 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:27.118371 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:27.118371 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:27.118371 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:27.118371 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:27.118371 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:27.118371 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:27.118371 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:27.118371 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:27.118371 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 14:07:27.118371 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 14:07:27.118371 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 14:07:27.118371 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 14:07:27.119451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:27.119451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:27.119451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:07:27.119451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:27.119451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:07:27.119451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:27.119451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:27.119451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:27.119451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:27.119451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:27.119451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:27.119451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:27.119451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:27.119451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:27.119451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:27.119451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:27.119451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:27.119451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:27.120633 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:27.120633 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:27.120633 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 14:07:27.120633 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 14:07:27.120633 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 14:07:27.120633 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 14:07:27.120633 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 14:07:27.120633 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 14:07:27.120633 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 05 14:07:27.120633 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:27.124570 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 295/615] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:07:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:27.129572 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec2be3-ed2f-4f8a-8e07-758b719acbff None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:27.143441 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-da6c46d4-22c1-4b45-89af-55cda4aa252a None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:27.145517 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-da6c46d4-22c1-4b45-89af-55cda4aa252a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:27.218272 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 321/616] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 14:07:27.293786 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec2be3-ed2f-4f8a-8e07-758b719acbff None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:27.298119 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c46d4-22c1-4b45-89af-55cda4aa252a None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:27.342469 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc61f38-b323-4231-b814-9799e69bc0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:27.343012 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:27.343129 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc61f38-b323-4231-b814-9799e69bc0e9 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:27.343353 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc61f38-b323-4231-b814-9799e69bc0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:27.343620 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc61f38-b323-4231-b814-9799e69bc0e9 None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:27.345625 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc61f38-b323-4231-b814-9799e69bc0e9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:27.345938 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc61f38-b323-4231-b814-9799e69bc0e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:27.347370 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc61f38-b323-4231-b814-9799e69bc0e9 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:27.393943 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 296/617] 10.222.0.26 () {56 vars in 1137 bytes} [Fri Aug 5 14:07:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:27.402842 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc61f38-b323-4231-b814-9799e69bc0e9 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:27.487829 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:27.512353 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c46d4-22c1-4b45-89af-55cda4aa252a None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:27.559515 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc61f38-b323-4231-b814-9799e69bc0e9 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:27.578705 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826bee67-3f1f-4f12-84f3-c4cb7ed15e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:27.579347 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826bee67-3f1f-4f12-84f3-c4cb7ed15e56 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:27.579619 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826bee67-3f1f-4f12-84f3-c4cb7ed15e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:27.579884 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826bee67-3f1f-4f12-84f3-c4cb7ed15e56 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:27.602955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826bee67-3f1f-4f12-84f3-c4cb7ed15e56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:07:27.607911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-826bee67-3f1f-4f12-84f3-c4cb7ed15e56 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:07:27.607911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:27.607911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:27.607911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:27.607911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:27.607911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:27.607911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:27.607911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:27.607911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:27.607911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:27.607911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:27.607911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:27.607911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:07:27.607911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:07:27.607911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:07:27.607911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:07:27.608862 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:07:27.608862 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:07:27.608862 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:07:27.608862 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:07:27.608862 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:27.608862 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:27.608862 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:27.608862 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:27.608862 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:27.608862 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:27.608862 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:27.608862 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:27.608862 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:27.608862 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:27.608862 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:27.608862 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:27.609704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:27.609704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:27.609704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:27.609704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:27.609704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:07:27.609704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:07:27.609704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:07:27.609704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:07:27.609704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:07:27.609704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:07:27.609704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:07:27.609704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:27.614272 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 322/618] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Aug 5 14:07:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:07:27.615208 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826bee67-3f1f-4f12-84f3-c4cb7ed15e56 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:27.635726 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c2e4bd-04b4-43a8-965b-205bf98200a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:27.636345 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2e4bd-04b4-43a8-965b-205bf98200a6 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:27.636643 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2e4bd-04b4-43a8-965b-205bf98200a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:27.636908 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2e4bd-04b4-43a8-965b-205bf98200a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:27.723597 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c2e4bd-04b4-43a8-965b-205bf98200a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:27.725105 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c2e4bd-04b4-43a8-965b-205bf98200a6 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:27.731805 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:27.762488 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826bee67-3f1f-4f12-84f3-c4cb7ed15e56 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:27.801515 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:27.807375 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052b098e-6edb-43db-b95c-50896c85c46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:27.808004 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b098e-6edb-43db-b95c-50896c85c46e None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:27.808278 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b098e-6edb-43db-b95c-50896c85c46e None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:27.808624 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b098e-6edb-43db-b95c-50896c85c46e None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:27.809350 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052b098e-6edb-43db-b95c-50896c85c46e None admin] RBAC: Adding query filter params (name=service) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:27.809647 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052b098e-6edb-43db-b95c-50896c85c46e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:27.811072 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052b098e-6edb-43db-b95c-50896c85c46e None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:27.822592 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 323/619] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Aug 5 14:07:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:27.823961 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c0f97-ed60-42a1-9eee-a7573d8ad5d6 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:27.824244 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c0f97-ed60-42a1-9eee-a7573d8ad5d6 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:27.824556 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c0f97-ed60-42a1-9eee-a7573d8ad5d6 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:27.826023 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 324/620] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:27.827615 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c0f97-ed60-42a1-9eee-a7573d8ad5d6 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:27.846728 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 297/621] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:07:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 14:07:27.849232 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffa1c3-6a39-4757-8461-fc5ae6404dca None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:27.849519 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffa1c3-6a39-4757-8461-fc5ae6404dca None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:27.849795 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffa1c3-6a39-4757-8461-fc5ae6404dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:27.873025 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-e0ffa1c3-6a39-4757-8461-fc5ae6404dca None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:27.874248 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e0ffa1c3-6a39-4757-8461-fc5ae6404dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:27.960525 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 298/622] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:07:27.961399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ffa1c3-6a39-4757-8461-fc5ae6404dca None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:27.972278 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c0f97-ed60-42a1-9eee-a7573d8ad5d6 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:28.017708 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77957177-f734-4624-bd54-dd20b5859f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:28.018264 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77957177-f734-4624-bd54-dd20b5859f69 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:28.018539 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77957177-f734-4624-bd54-dd20b5859f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:28.018808 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77957177-f734-4624-bd54-dd20b5859f69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:28.027811 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:28.032727 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77957177-f734-4624-bd54-dd20b5859f69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:28.034792 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77957177-f734-4624-bd54-dd20b5859f69 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:28.052782 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-77957177-f734-4624-bd54-dd20b5859f69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:28.052782 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:28.052782 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:28.052782 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:28.052782 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:28.052782 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:28.052782 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:28.052782 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:28.052782 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:28.052782 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:28.052782 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:28.052782 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:28.052782 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:28.052782 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:28.052782 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:28.053827 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:28.053827 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:28.053827 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:28.053827 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:28.053827 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:28.053827 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:28.053827 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:28.053827 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:28.053827 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:28.053827 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:28.053827 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:28.053827 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:28.053827 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:28.053827 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:28.053827 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:28.053827 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:28.053827 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:28.054824 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:28.054824 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:28.054824 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:28.054824 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:28.057379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 325/623] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:28.058499 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77957177-f734-4624-bd54-dd20b5859f69 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:28.082375 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:28.098212 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ffa1c3-6a39-4757-8461-fc5ae6404dca None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:28.142358 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e801e70-fca5-4b39-a5bb-62179a08c89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:28.143223 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e801e70-fca5-4b39-a5bb-62179a08c89e None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:28.143507 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e801e70-fca5-4b39-a5bb-62179a08c89e None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:28.143781 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e801e70-fca5-4b39-a5bb-62179a08c89e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:28.158320 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e801e70-fca5-4b39-a5bb-62179a08c89e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:28.161261 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e801e70-fca5-4b39-a5bb-62179a08c89e None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:28.181384 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-9e801e70-fca5-4b39-a5bb-62179a08c89e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:28.181384 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:28.181384 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:28.181384 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:28.181384 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:28.181384 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:28.181384 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:28.181384 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:28.181384 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:28.181384 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:28.181384 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:28.181384 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:28.181384 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:28.181384 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:28.181384 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:28.182499 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:28.182499 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:28.182499 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:28.182499 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:28.182499 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:28.182499 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:28.182499 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:28.182499 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:28.182499 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:28.182499 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:28.182499 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:28.182499 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:28.182499 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:28.182499 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:28.182499 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:28.182499 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:28.182499 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:28.183587 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:28.183587 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:28.183587 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:28.183587 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:28.183587 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 299/624] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:28.183938 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e801e70-fca5-4b39-a5bb-62179a08c89e None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:28.187648 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77957177-f734-4624-bd54-dd20b5859f69 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:28.235369 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf32709-9740-466d-ba8a-f96c7da7c45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:28.236019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf32709-9740-466d-ba8a-f96c7da7c45a None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:28.236296 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf32709-9740-466d-ba8a-f96c7da7c45a None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:28.236595 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf32709-9740-466d-ba8a-f96c7da7c45a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:28.250933 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf32709-9740-466d-ba8a-f96c7da7c45a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:07:28.260214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-ccf32709-9740-466d-ba8a-f96c7da7c45a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:07:28.260214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:28.260214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:28.260214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:28.260214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:28.260214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:28.260214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:28.260214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:28.260214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:28.260214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:28.260214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:28.260214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:28.260214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:07:28.260214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:07:28.260214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:07:28.260214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:07:28.261216 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:07:28.261216 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:07:28.261216 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:07:28.261216 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:07:28.261216 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:28.261216 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:28.261216 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:28.261216 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:28.261216 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:28.261216 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:28.261216 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:28.261216 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:28.261216 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:28.261216 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:28.261216 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:28.261216 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:28.262041 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:28.262041 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:28.262041 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:28.262041 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:28.262041 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:07:28.262041 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:07:28.262041 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:07:28.262041 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:07:28.262041 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:07:28.262041 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:07:28.262041 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:07:28.262041 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:28.264175 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 326/625] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Aug 5 14:07:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:28.265179 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf32709-9740-466d-ba8a-f96c7da7c45a None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:28.312923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e801e70-fca5-4b39-a5bb-62179a08c89e None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:28.330019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:28.352818 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d3b48-357e-4d16-8b3e-cecf84e41f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:28.353867 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d3b48-357e-4d16-8b3e-cecf84e41f01 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:28.354349 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d3b48-357e-4d16-8b3e-cecf84e41f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:28.354863 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d3b48-357e-4d16-8b3e-cecf84e41f01 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:28.355764 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0d3b48-357e-4d16-8b3e-cecf84e41f01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:28.356265 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0d3b48-357e-4d16-8b3e-cecf84e41f01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:28.357972 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0d3b48-357e-4d16-8b3e-cecf84e41f01 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:28.368380 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:28.372845 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 300/626] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:28.373875 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d3b48-357e-4d16-8b3e-cecf84e41f01 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:28.387353 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf32709-9740-466d-ba8a-f96c7da7c45a None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:28.424029 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c8703b-a6c2-475a-b342-02f232a8df9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:28.424940 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c8703b-a6c2-475a-b342-02f232a8df9c None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:28.425370 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c8703b-a6c2-475a-b342-02f232a8df9c None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:28.426411 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c8703b-a6c2-475a-b342-02f232a8df9c None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:28.427753 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c8703b-a6c2-475a-b342-02f232a8df9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:28.428328 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c8703b-a6c2-475a-b342-02f232a8df9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:28.430003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c8703b-a6c2-475a-b342-02f232a8df9c None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:28.448877 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 327/627] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:28.450043 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c8703b-a6c2-475a-b342-02f232a8df9c None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:28.479361 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d3b48-357e-4d16-8b3e-cecf84e41f01 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:28.525129 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce430958-ce23-4556-8ca0-d0e5d7dcd3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:28.525978 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce430958-ce23-4556-8ca0-d0e5d7dcd3bd None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:28.526398 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce430958-ce23-4556-8ca0-d0e5d7dcd3bd None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:28.526806 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce430958-ce23-4556-8ca0-d0e5d7dcd3bd None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:28.527696 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce430958-ce23-4556-8ca0-d0e5d7dcd3bd None admin] RBAC: Adding query filter params (name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:28.528148 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce430958-ce23-4556-8ca0-d0e5d7dcd3bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:28.529685 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce430958-ce23-4556-8ca0-d0e5d7dcd3bd None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:28.539055 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 301/628] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Aug 5 14:07:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:28.540070 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce430958-ce23-4556-8ca0-d0e5d7dcd3bd None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:28.562075 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c8703b-a6c2-475a-b342-02f232a8df9c None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:28.579833 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:28.600746 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e54824-fa13-4a34-afe7-2539623fc02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:28.601651 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e54824-fa13-4a34-afe7-2539623fc02e None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:28.602037 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e54824-fa13-4a34-afe7-2539623fc02e None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:28.603432 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e54824-fa13-4a34-afe7-2539623fc02e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:28.603851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:28.611269 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e54824-fa13-4a34-afe7-2539623fc02e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:28.613021 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e54824-fa13-4a34-afe7-2539623fc02e None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:28.627045 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-95e54824-fa13-4a34-afe7-2539623fc02e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 14:07:28.627045 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:28.627045 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:28.627045 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:28.627045 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:28.627045 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:28.627045 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:28.627045 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:28.627045 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:28.627045 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:28.627045 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:28.627045 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:28.627045 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 14:07:28.627045 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 14:07:28.627045 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 14:07:28.627045 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 14:07:28.633019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:28.633019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:28.633019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:07:28.633019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:28.633019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:07:28.633019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:28.633019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:28.633019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:28.633019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:28.633019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:28.633019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:28.633019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:28.633019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:28.633019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:28.633019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:28.633019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:28.633019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:28.633019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:28.634015 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:28.634015 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:28.634015 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 14:07:28.634015 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 14:07:28.634015 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 14:07:28.634015 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 14:07:28.634015 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 14:07:28.634015 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 14:07:28.634015 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 14:07:28.634015 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:28.634015 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 328/629] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Aug 5 14:07:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:28.634015 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e54824-fa13-4a34-afe7-2539623fc02e None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:28.645864 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce430958-ce23-4556-8ca0-d0e5d7dcd3bd None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:28.681133 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31755350-ecb1-447b-999c-eb1a7acb0e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:28.682566 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31755350-ecb1-447b-999c-eb1a7acb0e47 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:28.683304 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31755350-ecb1-447b-999c-eb1a7acb0e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:28.684225 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31755350-ecb1-447b-999c-eb1a7acb0e47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:28.693298 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31755350-ecb1-447b-999c-eb1a7acb0e47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:28.695080 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31755350-ecb1-447b-999c-eb1a7acb0e47 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:28.709424 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-31755350-ecb1-447b-999c-eb1a7acb0e47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:28.709424 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:28.709424 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:28.709424 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:28.709424 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:28.709424 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:28.709424 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:28.709424 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:28.709424 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:28.709424 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:28.709424 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:28.709424 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:28.709424 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 14:07:28.709424 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 14:07:28.709424 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 14:07:28.710373 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 14:07:28.710373 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:28.710373 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:28.710373 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:28.710373 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:28.710373 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:28.710373 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:28.710373 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:28.710373 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:28.710373 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:28.710373 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:28.710373 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:28.710373 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:28.710373 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:28.710373 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:28.710373 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:28.710373 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:28.711292 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 14:07:28.711292 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 14:07:28.711292 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:07:28.711292 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:07:28.711292 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:07:28.711292 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:07:28.711292 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:28.711292 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:28.719201 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 302/630] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:07:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:28.720297 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31755350-ecb1-447b-999c-eb1a7acb0e47 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:28.752916 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e54824-fa13-4a34-afe7-2539623fc02e None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:28.789085 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78feab4f-bd68-4c79-8e68-603c60a8b247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:28.789730 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78feab4f-bd68-4c79-8e68-603c60a8b247 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:28.790008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78feab4f-bd68-4c79-8e68-603c60a8b247 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:28.790348 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78feab4f-bd68-4c79-8e68-603c60a8b247 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:28.802724 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78feab4f-bd68-4c79-8e68-603c60a8b247 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:28.804163 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78feab4f-bd68-4c79-8e68-603c60a8b247 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:28.819561 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-78feab4f-bd68-4c79-8e68-603c60a8b247 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:28.819561 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:28.819561 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:28.819561 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:28.819561 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:28.819561 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:28.819561 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:28.819561 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:28.819561 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:28.819561 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:28.819561 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:28.819561 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:28.819561 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:28.819561 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:28.819561 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:28.820537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:28.820537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:28.820537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:28.820537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:28.820537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:28.820537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:28.820537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:28.820537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:28.820537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:28.820537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:28.820537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:28.820537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:28.820537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:28.820537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:28.820537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:28.820537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:28.820537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:28.822129 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:28.822129 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:28.822129 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:28.822129 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:28.822129 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 329/631] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:28.822129 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78feab4f-bd68-4c79-8e68-603c60a8b247 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:28.841079 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31755350-ecb1-447b-999c-eb1a7acb0e47 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:28.850033 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:28.864578 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:28.876579 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae308f95-4c39-4ac7-822e-097fc936e900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:28.877197 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae308f95-4c39-4ac7-822e-097fc936e900 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:28.877474 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae308f95-4c39-4ac7-822e-097fc936e900 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:28.878170 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae308f95-4c39-4ac7-822e-097fc936e900 None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:28.879887 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae308f95-4c39-4ac7-822e-097fc936e900 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:28.879887 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae308f95-4c39-4ac7-822e-097fc936e900 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:28.881368 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae308f95-4c39-4ac7-822e-097fc936e900 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:28.904395 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 303/632] 10.222.0.26 () {56 vars in 1135 bytes} [Fri Aug 5 14:07:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:28.904395 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea91bd-e220-40fe-b763-f2df5c3c1a72 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:28.904395 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea91bd-e220-40fe-b763-f2df5c3c1a72 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:28.904395 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea91bd-e220-40fe-b763-f2df5c3c1a72 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:28.904395 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 304/633] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:28.904395 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea91bd-e220-40fe-b763-f2df5c3c1a72 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:28.934592 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78feab4f-bd68-4c79-8e68-603c60a8b247 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:28.969350 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab301043-e7fe-4ed9-9d35-8b8dae7d36c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:28.969968 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab301043-e7fe-4ed9-9d35-8b8dae7d36c1 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:28.970246 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab301043-e7fe-4ed9-9d35-8b8dae7d36c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:28.970517 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab301043-e7fe-4ed9-9d35-8b8dae7d36c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:28.971285 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab301043-e7fe-4ed9-9d35-8b8dae7d36c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:28.971589 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab301043-e7fe-4ed9-9d35-8b8dae7d36c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:28.973599 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab301043-e7fe-4ed9-9d35-8b8dae7d36c1 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:28.986114 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 330/634] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Aug 5 14:07:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:28.986910 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab301043-e7fe-4ed9-9d35-8b8dae7d36c1 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:29.019454 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea91bd-e220-40fe-b763-f2df5c3c1a72 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:29.057317 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c5e79b-3dab-43e2-ba91-13c2f14dab40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:29.058243 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5e79b-3dab-43e2-ba91-13c2f14dab40 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:29.058701 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5e79b-3dab-43e2-ba91-13c2f14dab40 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:29.059160 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5e79b-3dab-43e2-ba91-13c2f14dab40 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:29.059917 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c5e79b-3dab-43e2-ba91-13c2f14dab40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:29.060220 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c5e79b-3dab-43e2-ba91-13c2f14dab40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:29.061629 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c5e79b-3dab-43e2-ba91-13c2f14dab40 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:29.077578 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 305/635] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:29.077578 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130fa22c-0e87-4c94-851a-18ada4094dc2 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:29.077578 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130fa22c-0e87-4c94-851a-18ada4094dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:29.077578 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130fa22c-0e87-4c94-851a-18ada4094dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:29.081108 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:29.093493 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-130fa22c-0e87-4c94-851a-18ada4094dc2 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:29.094611 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-130fa22c-0e87-4c94-851a-18ada4094dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:29.103415 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab301043-e7fe-4ed9-9d35-8b8dae7d36c1 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:29.133106 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:29.145735 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd359c0-bf48-4bd9-b266-a7228e16a3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:29.146727 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd359c0-bf48-4bd9-b266-a7228e16a3eb None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:29.147021 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd359c0-bf48-4bd9-b266-a7228e16a3eb None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:29.147292 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd359c0-bf48-4bd9-b266-a7228e16a3eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:29.161783 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd359c0-bf48-4bd9-b266-a7228e16a3eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:29.163738 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd359c0-bf48-4bd9-b266-a7228e16a3eb None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:29.176442 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 306/636] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:07:29.176442 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130fa22c-0e87-4c94-851a-18ada4094dc2 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:29.187089 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-0cd359c0-bf48-4bd9-b266-a7228e16a3eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:29.187089 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:29.187089 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:29.187089 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:29.187089 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:29.187089 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:29.187089 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:29.187089 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:29.187089 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:29.187089 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:29.187089 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:29.187089 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:29.187089 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:29.187089 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:29.187089 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:29.188012 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:29.188012 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:29.188012 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:29.188012 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:29.188012 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:29.188012 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:29.188012 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:29.188012 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:29.188012 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:29.188012 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:29.188012 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:29.188012 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:29.188012 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:29.188012 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:29.188012 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:29.188012 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:29.188012 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:29.188945 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:29.188945 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:29.188945 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:29.188945 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:29.189165 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 331/637] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:29.189826 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd359c0-bf48-4bd9-b266-a7228e16a3eb None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:29.281901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130fa22c-0e87-4c94-851a-18ada4094dc2 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:29.292791 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd359c0-bf48-4bd9-b266-a7228e16a3eb None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:29.328954 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36601e4b-33d0-4435-9174-eb48069ff95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:29.329614 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36601e4b-33d0-4435-9174-eb48069ff95e None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:29.329892 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36601e4b-33d0-4435-9174-eb48069ff95e None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:29.330163 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36601e4b-33d0-4435-9174-eb48069ff95e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:29.332609 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:29.339560 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36601e4b-33d0-4435-9174-eb48069ff95e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:29.341072 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36601e4b-33d0-4435-9174-eb48069ff95e None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:29.359503 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-36601e4b-33d0-4435-9174-eb48069ff95e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 14:07:29.359503 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:29.359503 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:29.359503 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:29.359503 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:29.359503 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:29.359503 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:29.359503 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:29.359503 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:29.359503 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:29.359503 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:29.359503 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:29.359503 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 14:07:29.359503 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 14:07:29.359503 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 14:07:29.359503 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 14:07:29.360615 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:29.360615 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:29.360615 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:07:29.360615 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:29.360615 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:07:29.360615 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:29.360615 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:29.360615 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:29.360615 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:29.360615 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:29.360615 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:29.360615 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:29.360615 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:29.360615 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:29.360615 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:29.360615 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:29.360615 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:29.360615 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:29.361565 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:29.361565 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:29.361565 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 14:07:29.361565 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 14:07:29.361565 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 14:07:29.361565 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 14:07:29.361565 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 14:07:29.361565 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 14:07:29.361565 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 14:07:29.361565 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:29.361565 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 332/638] 10.222.0.26 () {56 vars in 1093 bytes} [Fri Aug 5 14:07:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:29.361565 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36601e4b-33d0-4435-9174-eb48069ff95e None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:29.432581 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a8ddf-d158-4e53-aacb-0c23e3b5da2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:29.433193 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a8ddf-d158-4e53-aacb-0c23e3b5da2d None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:29.433465 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a8ddf-d158-4e53-aacb-0c23e3b5da2d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:29.433732 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a8ddf-d158-4e53-aacb-0c23e3b5da2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:29.434538 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a8ddf-d158-4e53-aacb-0c23e3b5da2d None admin] RBAC: Adding query filter params (role.id=ec18a2850fd540a39d4a935348bb4474, scope.project.id=319973e011c54b4ca1f2198c8a5e135b, user.id=dd1ce2a2b4eb405b999d3fb756795176) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:29.434842 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a8ddf-d158-4e53-aacb-0c23e3b5da2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:29.436259 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a8ddf-d158-4e53-aacb-0c23e3b5da2d None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:29.442139 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 307/639] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:07:29 2022] GET /identity/v3/role_assignments?user.id=dd1ce2a2b4eb405b999d3fb756795176&scope.project.id=319973e011c54b4ca1f2198c8a5e135b&role.id=ec18a2850fd540a39d4a935348bb4474 => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:29.442898 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a8ddf-d158-4e53-aacb-0c23e3b5da2d None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:29.459944 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36601e4b-33d0-4435-9174-eb48069ff95e None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:29.498177 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:29.501060 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c658a1-51cb-451e-b659-4caa4c5a37bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:29.501707 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c658a1-51cb-451e-b659-4caa4c5a37bb None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:29.501993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c658a1-51cb-451e-b659-4caa4c5a37bb None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:29.502265 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c658a1-51cb-451e-b659-4caa4c5a37bb None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:29.503006 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c658a1-51cb-451e-b659-4caa4c5a37bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:29.503420 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c658a1-51cb-451e-b659-4caa4c5a37bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:29.505425 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c658a1-51cb-451e-b659-4caa4c5a37bb None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:29.526215 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 333/640] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:29.527817 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c62da-cf2c-4679-b1f6-9ed46428278f None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:29.527950 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c62da-cf2c-4679-b1f6-9ed46428278f None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:29.528153 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c62da-cf2c-4679-b1f6-9ed46428278f None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:29.533444 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 334/641] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:29.533444 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c62da-cf2c-4679-b1f6-9ed46428278f None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:29.582527 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a8ddf-d158-4e53-aacb-0c23e3b5da2d None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:29.595983 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:29.621721 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ccfc8-c95d-4918-aba9-5a30413b6468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:29.622283 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ccfc8-c95d-4918-aba9-5a30413b6468 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:29.622556 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ccfc8-c95d-4918-aba9-5a30413b6468 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:29.622822 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ccfc8-c95d-4918-aba9-5a30413b6468 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:29.635519 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92ccfc8-c95d-4918-aba9-5a30413b6468 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:07:29.641412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-c92ccfc8-c95d-4918-aba9-5a30413b6468 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:07:29.641412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:29.641412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:29.641412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:29.641412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:29.641412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:29.641412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:29.641412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:29.641412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:29.641412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:29.641412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:29.641412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:29.641412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:07:29.641412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:07:29.641412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:07:29.641412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:07:29.642408 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:07:29.642408 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:07:29.642408 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:07:29.642408 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:07:29.642408 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:29.642408 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:29.642408 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:29.642408 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:29.642408 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:29.642408 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:29.642408 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:29.642408 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:29.642408 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:29.642408 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:29.642408 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:29.642408 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:29.643276 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:29.643276 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:29.643276 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:29.643276 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:29.643276 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:07:29.643276 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:07:29.643276 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:07:29.643276 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:07:29.643276 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:07:29.643276 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:07:29.643276 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:07:29.643276 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:29.643276 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 308/642] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Aug 5 14:07:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:29.644015 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ccfc8-c95d-4918-aba9-5a30413b6468 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:29.658197 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c62da-cf2c-4679-b1f6-9ed46428278f None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:29.697204 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f89de-2f1a-4e32-ba74-2296bb7e65df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:29.698149 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f89de-2f1a-4e32-ba74-2296bb7e65df None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:29.698713 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f89de-2f1a-4e32-ba74-2296bb7e65df None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:29.698990 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f89de-2f1a-4e32-ba74-2296bb7e65df None admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:29.699944 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f89de-2f1a-4e32-ba74-2296bb7e65df None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:29.700261 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f89de-2f1a-4e32-ba74-2296bb7e65df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:29.702481 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f89de-2f1a-4e32-ba74-2296bb7e65df None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:29.730430 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 335/643] 10.222.0.26 () {56 vars in 1133 bytes} [Fri Aug 5 14:07:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:29.730430 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab1f52-d265-495b-b6a0-f8f07d84c953 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:29.730430 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab1f52-d265-495b-b6a0-f8f07d84c953 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:29.730430 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab1f52-d265-495b-b6a0-f8f07d84c953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:29.749743 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:29.749988 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-81ab1f52-d265-495b-b6a0-f8f07d84c953 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:29.751357 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-81ab1f52-d265-495b-b6a0-f8f07d84c953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:29.756567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ccfc8-c95d-4918-aba9-5a30413b6468 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:29.790172 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb5149-a92d-4e6f-9717-d24e2a9369e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:29.790814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb5149-a92d-4e6f-9717-d24e2a9369e1 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:29.791091 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb5149-a92d-4e6f-9717-d24e2a9369e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:29.791363 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb5149-a92d-4e6f-9717-d24e2a9369e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:29.798223 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fb5149-a92d-4e6f-9717-d24e2a9369e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:29.799991 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fb5149-a92d-4e6f-9717-d24e2a9369e1 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:29.812399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-80fb5149-a92d-4e6f-9717-d24e2a9369e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:29.812399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:29.812399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:29.812399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:29.812399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:29.812399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:29.812399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:29.812399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:29.812399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:29.812399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:29.812399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:29.812399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:29.812399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 14:07:29.812399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 14:07:29.812399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 14:07:29.813349 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 14:07:29.813349 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:29.813349 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:29.813349 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:29.813349 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:29.813349 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:29.813349 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:29.813349 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:29.813349 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:29.813349 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:29.813349 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:29.813349 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:29.813349 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:29.813349 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:29.813349 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:29.813349 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:29.813349 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:29.814405 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 14:07:29.814405 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 14:07:29.814405 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:07:29.814405 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:07:29.814405 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:07:29.814405 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:07:29.814405 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:29.814405 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:29.818178 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 309/644] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:07:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:29.818956 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb5149-a92d-4e6f-9717-d24e2a9369e1 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:29.828258 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 336/645] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:07:29.829415 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab1f52-d265-495b-b6a0-f8f07d84c953 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:29.884020 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:29.931350 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb5149-a92d-4e6f-9717-d24e2a9369e1 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:29.942321 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab1f52-d265-495b-b6a0-f8f07d84c953 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:29.969366 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc65a7-d388-4f54-b5e0-ab764ae0609a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:29.970213 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc65a7-d388-4f54-b5e0-ab764ae0609a None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:29.970628 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc65a7-d388-4f54-b5e0-ab764ae0609a None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:29.971139 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc65a7-d388-4f54-b5e0-ab764ae0609a None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:29.972095 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bc65a7-d388-4f54-b5e0-ab764ae0609a None admin] RBAC: Adding query filter params (name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:29.972598 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bc65a7-d388-4f54-b5e0-ab764ae0609a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:29.974199 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bc65a7-d388-4f54-b5e0-ab764ae0609a None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:29.976333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c356900-9fa5-4e26-a52c-ec3cb8283bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:29.976997 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c356900-9fa5-4e26-a52c-ec3cb8283bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:29.977275 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c356900-9fa5-4e26-a52c-ec3cb8283bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:29.977562 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c356900-9fa5-4e26-a52c-ec3cb8283bd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:29.984436 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 310/646] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Aug 5 14:07:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:29.985399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc65a7-d388-4f54-b5e0-ab764ae0609a None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:30.005821 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:30.054590 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c356900-9fa5-4e26-a52c-ec3cb8283bd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:30.056036 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c356900-9fa5-4e26-a52c-ec3cb8283bd5 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:30.088192 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc65a7-d388-4f54-b5e0-ab764ae0609a None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:30.099564 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:30.122843 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75113a38-ab6d-4121-8ba4-c453575ce9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:30.123501 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75113a38-ab6d-4121-8ba4-c453575ce9d1 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:30.123862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75113a38-ab6d-4121-8ba4-c453575ce9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:30.124232 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75113a38-ab6d-4121-8ba4-c453575ce9d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:30.125040 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75113a38-ab6d-4121-8ba4-c453575ce9d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:30.125345 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75113a38-ab6d-4121-8ba4-c453575ce9d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:30.126750 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75113a38-ab6d-4121-8ba4-c453575ce9d1 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:30.140336 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 311/647] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Aug 5 14:07:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:30.141371 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75113a38-ab6d-4121-8ba4-c453575ce9d1 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:30.160769 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 337/648] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:07:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 14:07:30.161913 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c356900-9fa5-4e26-a52c-ec3cb8283bd5 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:30.237339 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:30.258971 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75113a38-ab6d-4121-8ba4-c453575ce9d1 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:30.265659 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c356900-9fa5-4e26-a52c-ec3cb8283bd5 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:30.295593 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6627c-91dc-450b-9ff6-360945bfb3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:30.296339 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6627c-91dc-450b-9ff6-360945bfb3ad None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:30.296646 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6627c-91dc-450b-9ff6-360945bfb3ad None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:30.296921 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6627c-91dc-450b-9ff6-360945bfb3ad None admin] PATH_INFO: `/v3/roles/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:30.300892 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a6816-280c-4798-bdc5-25c2e4a8a691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:30.301539 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a6816-280c-4798-bdc5-25c2e4a8a691 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:30.301812 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a6816-280c-4798-bdc5-25c2e4a8a691 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:30.302081 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a6816-280c-4798-bdc5-25c2e4a8a691 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:30.307412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6627c-91dc-450b-9ff6-360945bfb3ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:07:30.312088 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a6816-280c-4798-bdc5-25c2e4a8a691 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:30.312573 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-c4e6627c-91dc-450b-9ff6-360945bfb3ad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:07:30.312573 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:30.312573 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:30.312573 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:30.312573 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:30.312573 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:30.312573 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:30.312573 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:30.312573 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:30.312573 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:30.312573 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:30.312573 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:30.312573 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:07:30.312573 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:07:30.312573 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:07:30.312573 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:07:30.313498 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:07:30.313498 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:07:30.313498 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:07:30.313498 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:07:30.313498 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:30.313498 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:30.313498 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:30.313498 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:30.313498 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:30.313498 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:30.313498 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:30.313498 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:30.313498 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:30.313498 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:30.313498 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:30.313498 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:30.319682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:30.319682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:30.319682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:30.319682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:30.319682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:07:30.319682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:07:30.319682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:07:30.319682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:07:30.319682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:07:30.319682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:07:30.319682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:07:30.319682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:30.320376 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a6816-280c-4798-bdc5-25c2e4a8a691 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:30.320484 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 312/649] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Aug 5 14:07:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:30.320484 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6627c-91dc-450b-9ff6-360945bfb3ad None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:30.320610 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:30.335019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-bc7a6816-280c-4798-bdc5-25c2e4a8a691 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:30.335019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:30.335019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:30.335019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:30.335019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:30.335019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:30.335019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:30.335019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:30.335019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:30.335019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:30.335019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:30.335019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:30.335019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:30.335019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:30.335019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:30.336145 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:30.336145 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:30.336145 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:30.336145 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:30.336145 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:30.336145 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:30.336145 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:30.336145 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:30.336145 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:30.336145 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:30.336145 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:30.336145 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:30.336145 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:30.336145 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:30.336145 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:30.336145 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:30.336145 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:30.337364 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:30.337364 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:30.337364 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:30.337364 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:30.342443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 338/650] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:30.342443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a6816-280c-4798-bdc5-25c2e4a8a691 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:30.460214 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6627c-91dc-450b-9ff6-360945bfb3ad None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:30.495658 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a6816-280c-4798-bdc5-25c2e4a8a691 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:30.511160 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd14d644-22c1-4fa7-9c14-55ffa10060a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:30.512107 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd14d644-22c1-4fa7-9c14-55ffa10060a4 None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:30.512384 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd14d644-22c1-4fa7-9c14-55ffa10060a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:30.512749 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd14d644-22c1-4fa7-9c14-55ffa10060a4 None admin] PATH_INFO: `/v3/projects/319973e011c54b4ca1f2198c8a5e135b/users/dddd7eea67f34f0cbb5f6a1cd85f1fc5/roles/ec18a2850fd540a39d4a935348bb4474` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:30.517308 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:30.530470 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd14d644-22c1-4fa7-9c14-55ffa10060a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=319973e011c54b4ca1f2198c8a5e135b, user_id=dddd7eea67f34f0cbb5f6a1cd85f1fc5, role_id=ec18a2850fd540a39d4a935348bb4474)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:30.532596 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd14d644-22c1-4fa7-9c14-55ffa10060a4 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:30.544796 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213edb2-ccf8-4de1-8434-470454cf0f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:30.545560 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213edb2-ccf8-4de1-8434-470454cf0f16 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:30.545938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213edb2-ccf8-4de1-8434-470454cf0f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:30.546301 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213edb2-ccf8-4de1-8434-470454cf0f16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:30.556988 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c213edb2-ccf8-4de1-8434-470454cf0f16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:30.558388 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c213edb2-ccf8-4de1-8434-470454cf0f16 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:30.573273 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-c213edb2-ccf8-4de1-8434-470454cf0f16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:30.573273 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:30.573273 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:30.573273 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:30.573273 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:30.573273 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:30.573273 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:30.573273 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:30.573273 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:30.573273 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:30.573273 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:30.573273 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:30.573273 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:30.573273 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:30.573273 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:30.574397 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:30.574397 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:30.574397 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:30.574397 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:30.574397 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:30.574397 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:30.574397 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:30.574397 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:30.574397 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:30.574397 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:30.574397 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:30.574397 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:30.574397 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:30.574397 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:30.574397 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:30.574397 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:30.574397 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:30.575319 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:30.575319 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:30.575319 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:30.575319 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:30.575319 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 339/651] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:30.575591 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213edb2-ccf8-4de1-8434-470454cf0f16 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:30.622547 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 313/652] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Aug 5 14:07:30 2022] PUT /identity/v3/projects/319973e011c54b4ca1f2198c8a5e135b/users/dddd7eea67f34f0cbb5f6a1cd85f1fc5/roles/ec18a2850fd540a39d4a935348bb4474 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:07:30.623469 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd14d644-22c1-4fa7-9c14-55ffa10060a4 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:30.634091 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:30.697282 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213edb2-ccf8-4de1-8434-470454cf0f16 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:30.737192 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e451f-a0ca-483d-afdf-66e6002843b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:30.737782 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e451f-a0ca-483d-afdf-66e6002843b2 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:30.738723 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e451f-a0ca-483d-afdf-66e6002843b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:30.739129 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e451f-a0ca-483d-afdf-66e6002843b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:30.740016 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e451f-a0ca-483d-afdf-66e6002843b2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:30.740300 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e451f-a0ca-483d-afdf-66e6002843b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:30.741704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e451f-a0ca-483d-afdf-66e6002843b2 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:30.759540 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 340/653] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Aug 5 14:07:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:30.760287 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e451f-a0ca-483d-afdf-66e6002843b2 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:30.767184 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd14d644-22c1-4fa7-9c14-55ffa10060a4 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:30.818327 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e6f5b-87e7-4f02-bcb3-3624dff46309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:30.818978 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e6f5b-87e7-4f02-bcb3-3624dff46309 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:30.819217 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e6f5b-87e7-4f02-bcb3-3624dff46309 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:30.819486 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e6f5b-87e7-4f02-bcb3-3624dff46309 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:30.820180 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e6f5b-87e7-4f02-bcb3-3624dff46309 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:30.821085 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e6f5b-87e7-4f02-bcb3-3624dff46309 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:30.821903 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e6f5b-87e7-4f02-bcb3-3624dff46309 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:30.835988 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 314/654] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:30.841001 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e6f5b-87e7-4f02-bcb3-3624dff46309 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:30.887534 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e451f-a0ca-483d-afdf-66e6002843b2 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:30.898352 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:30.902873 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:30.925530 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393cfa9a-84e1-4dd6-8bc9-dd822a287704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:30.926951 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393cfa9a-84e1-4dd6-8bc9-dd822a287704 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:30.927624 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393cfa9a-84e1-4dd6-8bc9-dd822a287704 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:30.928238 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393cfa9a-84e1-4dd6-8bc9-dd822a287704 None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:30.929292 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393cfa9a-84e1-4dd6-8bc9-dd822a287704 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:30.929789 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393cfa9a-84e1-4dd6-8bc9-dd822a287704 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:30.931335 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393cfa9a-84e1-4dd6-8bc9-dd822a287704 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:30.945237 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 341/655] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:30.946270 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393cfa9a-84e1-4dd6-8bc9-dd822a287704 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:30.970494 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e6f5b-87e7-4f02-bcb3-3624dff46309 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:31.014974 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9eb21-710f-48ba-8568-f14a3b397507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:31.015951 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9eb21-710f-48ba-8568-f14a3b397507 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:31.016367 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9eb21-710f-48ba-8568-f14a3b397507 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:31.016795 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9eb21-710f-48ba-8568-f14a3b397507 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:31.029420 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c9eb21-710f-48ba-8568-f14a3b397507 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:31.031054 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c9eb21-710f-48ba-8568-f14a3b397507 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:31.047768 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-69c9eb21-710f-48ba-8568-f14a3b397507 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:31.047768 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:31.047768 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:31.047768 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:31.047768 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:31.047768 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:31.047768 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:31.047768 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:31.047768 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:31.047768 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:31.047768 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:31.047768 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:31.047768 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:31.047768 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:31.047768 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:31.048905 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:31.048905 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:31.048905 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:31.048905 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:31.048905 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:31.048905 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:31.048905 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:31.048905 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:31.048905 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:31.048905 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:31.048905 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:31.048905 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:31.048905 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:31.048905 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:31.048905 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:31.048905 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:31.048905 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:31.049985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:31.049985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:31.049985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:31.049985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:31.052988 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 315/656] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:31.054032 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9eb21-710f-48ba-8568-f14a3b397507 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:31.071493 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393cfa9a-84e1-4dd6-8bc9-dd822a287704 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:31.114763 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3709ee32-89ad-4d34-83f0-424a62a2a72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:31.115834 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3709ee32-89ad-4d34-83f0-424a62a2a72a None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:31.116205 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3709ee32-89ad-4d34-83f0-424a62a2a72a None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:31.116609 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3709ee32-89ad-4d34-83f0-424a62a2a72a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:31.125146 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3709ee32-89ad-4d34-83f0-424a62a2a72a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:31.126706 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3709ee32-89ad-4d34-83f0-424a62a2a72a None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:31.138967 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-3709ee32-89ad-4d34-83f0-424a62a2a72a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 14:07:31.138967 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:31.138967 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:31.138967 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:31.138967 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:31.138967 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:31.138967 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:31.138967 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:31.138967 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:31.138967 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:31.138967 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:31.138967 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:31.138967 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 14:07:31.138967 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 14:07:31.138967 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 14:07:31.138967 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 14:07:31.140134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:31.140134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:31.140134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:07:31.140134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:31.140134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:07:31.140134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:31.140134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:31.140134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:31.140134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:31.140134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:31.140134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:31.140134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:31.140134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:31.140134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:31.140134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:31.140134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:31.140134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:31.140134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:31.141296 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:31.141296 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:31.141296 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 14:07:31.141296 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 14:07:31.141296 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 14:07:31.141296 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 14:07:31.141296 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 14:07:31.141296 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 14:07:31.141296 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 14:07:31.141296 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:31.147504 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 342/657] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:07:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:31.148586 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3709ee32-89ad-4d34-83f0-424a62a2a72a None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:31.168672 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:31.173351 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9eb21-710f-48ba-8568-f14a3b397507 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:31.187082 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:31.217878 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e09c76-660d-4977-8d57-4f3f69c0e017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:31.218799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e09c76-660d-4977-8d57-4f3f69c0e017 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:31.219259 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e09c76-660d-4977-8d57-4f3f69c0e017 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:31.219682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e09c76-660d-4977-8d57-4f3f69c0e017 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:31.227910 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e09c76-660d-4977-8d57-4f3f69c0e017 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:31.229638 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e09c76-660d-4977-8d57-4f3f69c0e017 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:31.243862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-e2e09c76-660d-4977-8d57-4f3f69c0e017 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:31.243862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:31.243862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:31.243862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:31.243862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:31.243862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:31.243862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:31.243862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:31.243862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:31.243862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:31.243862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:31.243862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:31.243862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 14:07:31.243862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 14:07:31.243862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 14:07:31.244815 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 14:07:31.244815 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:31.244815 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:31.244815 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:31.244815 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:31.244815 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:31.244815 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:31.244815 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:31.244815 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:31.244815 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:31.244815 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:31.244815 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:31.244815 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:31.244815 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:31.244815 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:31.244815 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:31.244815 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:31.245729 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 14:07:31.245729 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 14:07:31.245729 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:07:31.245729 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:07:31.245729 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:07:31.245729 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:07:31.245729 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:31.245729 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:31.248027 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 316/658] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:07:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:31.249048 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e09c76-660d-4977-8d57-4f3f69c0e017 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:31.287085 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3709ee32-89ad-4d34-83f0-424a62a2a72a None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:31.330729 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68c41e-8c19-4d53-a022-5c2dce1d5498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:31.331352 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68c41e-8c19-4d53-a022-5c2dce1d5498 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:31.331632 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68c41e-8c19-4d53-a022-5c2dce1d5498 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:31.332450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68c41e-8c19-4d53-a022-5c2dce1d5498 None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:31.333277 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d68c41e-8c19-4d53-a022-5c2dce1d5498 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:31.333584 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d68c41e-8c19-4d53-a022-5c2dce1d5498 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:31.335324 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d68c41e-8c19-4d53-a022-5c2dce1d5498 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:31.353181 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 343/659] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:31.354939 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68c41e-8c19-4d53-a022-5c2dce1d5498 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:31.368165 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e09c76-660d-4977-8d57-4f3f69c0e017 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:31.406271 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf7d1c-a1cb-49cd-92cb-f769e6005cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:31.407372 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf7d1c-a1cb-49cd-92cb-f769e6005cda None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:31.407920 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf7d1c-a1cb-49cd-92cb-f769e6005cda None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:31.408388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf7d1c-a1cb-49cd-92cb-f769e6005cda None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:31.409461 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cf7d1c-a1cb-49cd-92cb-f769e6005cda None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:31.410592 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cf7d1c-a1cb-49cd-92cb-f769e6005cda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:31.413055 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cf7d1c-a1cb-49cd-92cb-f769e6005cda None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:31.433647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 317/660] 10.222.0.26 () {56 vars in 1137 bytes} [Fri Aug 5 14:07:31 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:31.434793 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf7d1c-a1cb-49cd-92cb-f769e6005cda None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:31.455245 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:31.455828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:31.468383 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68c41e-8c19-4d53-a022-5c2dce1d5498 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:31.505281 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c256d4-1e4e-449d-977b-9b4976e94a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:31.506153 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c256d4-1e4e-449d-977b-9b4976e94a0e None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:31.506431 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c256d4-1e4e-449d-977b-9b4976e94a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:31.506707 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c256d4-1e4e-449d-977b-9b4976e94a0e None admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:31.507513 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c256d4-1e4e-449d-977b-9b4976e94a0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:31.507761 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c256d4-1e4e-449d-977b-9b4976e94a0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:31.510199 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c256d4-1e4e-449d-977b-9b4976e94a0e None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:31.528480 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 344/661] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Aug 5 14:07:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:31.529558 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c256d4-1e4e-449d-977b-9b4976e94a0e None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:31.564368 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf7d1c-a1cb-49cd-92cb-f769e6005cda None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:31.604888 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c09f0-23ae-477d-ad5b-357437e92a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:31.605523 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c09f0-23ae-477d-ad5b-357437e92a20 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:31.605796 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c09f0-23ae-477d-ad5b-357437e92a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:31.606063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c09f0-23ae-477d-ad5b-357437e92a20 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:31.614325 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c09f0-23ae-477d-ad5b-357437e92a20 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:31.615736 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c09f0-23ae-477d-ad5b-357437e92a20 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:31.627623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-8f2c09f0-23ae-477d-ad5b-357437e92a20 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 14:07:31.627623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:31.627623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:31.627623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:31.627623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:31.627623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:31.627623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:31.627623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:31.627623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:31.627623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:31.627623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:31.627623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:31.627623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 14:07:31.627623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 14:07:31.627623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 14:07:31.627623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 14:07:31.629567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:31.629567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:31.629567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:07:31.629567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:31.629567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:07:31.629567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:31.629567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:31.629567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:31.629567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:31.629567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:31.629567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:31.629567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:31.629567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:31.629567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:31.629567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:31.629567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:31.629567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:31.629567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:31.630974 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:31.630974 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:31.630974 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 14:07:31.630974 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 14:07:31.630974 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 14:07:31.630974 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 14:07:31.630974 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 14:07:31.630974 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 14:07:31.630974 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 14:07:31.630974 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:31.633788 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 318/662] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Aug 5 14:07:31 2022] GET /identity/v3/users/neutron => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:31.634644 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c09f0-23ae-477d-ad5b-357437e92a20 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:31.645712 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c256d4-1e4e-449d-977b-9b4976e94a0e None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:31.680689 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0506d3-1ef9-4a7f-95c1-beb201e2751f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:31.681307 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0506d3-1ef9-4a7f-95c1-beb201e2751f None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:31.681581 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0506d3-1ef9-4a7f-95c1-beb201e2751f None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:31.681848 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0506d3-1ef9-4a7f-95c1-beb201e2751f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:31.700243 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:31.718076 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:31.742420 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c09f0-23ae-477d-ad5b-357437e92a20 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:31.758478 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0506d3-1ef9-4a7f-95c1-beb201e2751f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:31.760052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0506d3-1ef9-4a7f-95c1-beb201e2751f None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:31.777386 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d00ce-8d37-4fad-842c-a1a5ce294391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:31.777995 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d00ce-8d37-4fad-842c-a1a5ce294391 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:31.778269 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d00ce-8d37-4fad-842c-a1a5ce294391 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:31.778559 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d00ce-8d37-4fad-842c-a1a5ce294391 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:31.779424 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d00ce-8d37-4fad-842c-a1a5ce294391 None admin] RBAC: Adding query filter params (role.id=ec18a2850fd540a39d4a935348bb4474, scope.project.id=319973e011c54b4ca1f2198c8a5e135b, user.id=5ec4940c99fc4fa8a5bc7cdabc4558fb) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:31.779728 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d00ce-8d37-4fad-842c-a1a5ce294391 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:31.781186 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d00ce-8d37-4fad-842c-a1a5ce294391 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:31.796658 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 319/663] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:07:31 2022] GET /identity/v3/role_assignments?user.id=5ec4940c99fc4fa8a5bc7cdabc4558fb&scope.project.id=319973e011c54b4ca1f2198c8a5e135b&role.id=ec18a2850fd540a39d4a935348bb4474 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:31.798355 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2919d9-dce8-417a-8f41-25defaa6e782 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:31.798355 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2919d9-dce8-417a-8f41-25defaa6e782 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:31.798641 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2919d9-dce8-417a-8f41-25defaa6e782 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:31.800087 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 320/664] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:31.808763 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2919d9-dce8-417a-8f41-25defaa6e782 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:31.892611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 345/665] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:07:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 14:07:31.893974 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ed23a-ec5c-4353-9cee-18c93cde7325 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:31.894352 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ed23a-ec5c-4353-9cee-18c93cde7325 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:31.895165 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ed23a-ec5c-4353-9cee-18c93cde7325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:31.918109 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-fd2ed23a-ec5c-4353-9cee-18c93cde7325 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:31.919649 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-fd2ed23a-ec5c-4353-9cee-18c93cde7325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:31.936016 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2919d9-dce8-417a-8f41-25defaa6e782 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:31.978040 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:31.978452 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3490b01f-e9ca-43b1-b8af-fd2d7118da2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:31.979955 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490b01f-e9ca-43b1-b8af-fd2d7118da2b None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:31.980576 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490b01f-e9ca-43b1-b8af-fd2d7118da2b None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:31.981043 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490b01f-e9ca-43b1-b8af-fd2d7118da2b None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:31.982795 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3490b01f-e9ca-43b1-b8af-fd2d7118da2b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:31.983287 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3490b01f-e9ca-43b1-b8af-fd2d7118da2b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:31.984917 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3490b01f-e9ca-43b1-b8af-fd2d7118da2b None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:31.995301 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:32.005598 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 321/666] 10.222.0.26 () {56 vars in 1139 bytes} [Fri Aug 5 14:07:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:32.006541 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3490b01f-e9ca-43b1-b8af-fd2d7118da2b None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:32.007295 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 346/667] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:07:32.011066 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ed23a-ec5c-4353-9cee-18c93cde7325 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:32.144092 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3490b01f-e9ca-43b1-b8af-fd2d7118da2b None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:32.148526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ed23a-ec5c-4353-9cee-18c93cde7325 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:32.190391 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25653f-af7d-4ce7-8834-d5cac9311476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:32.191052 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25653f-af7d-4ce7-8834-d5cac9311476 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:32.191341 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25653f-af7d-4ce7-8834-d5cac9311476 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:32.191613 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25653f-af7d-4ce7-8834-d5cac9311476 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:32.192878 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8052bd-e638-45bb-90b4-9155d373a7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:32.193881 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8052bd-e638-45bb-90b4-9155d373a7db None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:32.194685 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8052bd-e638-45bb-90b4-9155d373a7db None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:32.195158 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8052bd-e638-45bb-90b4-9155d373a7db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:32.203143 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25653f-af7d-4ce7-8834-d5cac9311476 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:32.204612 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25653f-af7d-4ce7-8834-d5cac9311476 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:32.223612 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-df25653f-af7d-4ce7-8834-d5cac9311476 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:32.223612 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:32.223612 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:32.223612 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:32.223612 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:32.223612 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:32.223612 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:32.223612 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:32.223612 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:32.223612 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:32.223612 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:32.223612 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:32.223612 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:32.223612 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:32.223612 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:32.224609 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:32.224609 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:32.224609 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:32.224609 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:32.224609 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:32.224609 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:32.224609 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:32.224609 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:32.224609 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:32.224609 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:32.224609 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:32.224609 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:32.224609 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:32.224609 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:32.224609 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:32.224609 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:32.224609 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:32.225498 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:32.225498 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:32.225498 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:32.225498 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:32.225498 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 322/668] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:32.226201 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25653f-af7d-4ce7-8834-d5cac9311476 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:32.278189 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8052bd-e638-45bb-90b4-9155d373a7db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:32.280525 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8052bd-e638-45bb-90b4-9155d373a7db None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:32.284801 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:32.290324 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:32.347802 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25653f-af7d-4ce7-8834-d5cac9311476 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:32.390940 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 347/669] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:07:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 14:07:32.391936 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8052bd-e638-45bb-90b4-9155d373a7db None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:32.393755 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde15dbd-0e75-450b-93a3-94eb8b82f6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:32.395478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde15dbd-0e75-450b-93a3-94eb8b82f6f9 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:32.396780 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde15dbd-0e75-450b-93a3-94eb8b82f6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:32.400775 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde15dbd-0e75-450b-93a3-94eb8b82f6f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:32.417306 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde15dbd-0e75-450b-93a3-94eb8b82f6f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:32.420038 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde15dbd-0e75-450b-93a3-94eb8b82f6f9 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:32.440268 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-bde15dbd-0e75-450b-93a3-94eb8b82f6f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:32.440268 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:32.440268 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:32.440268 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:32.440268 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:32.440268 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:32.440268 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:32.440268 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:32.440268 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:32.440268 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:32.440268 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:32.440268 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:32.440268 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:32.440268 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:32.440268 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:32.441418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:32.441418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:32.441418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:32.441418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:32.441418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:32.441418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:32.441418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:32.441418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:32.441418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:32.441418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:32.441418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:32.441418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:32.441418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:32.441418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:32.441418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:32.441418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:32.441418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:32.442480 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:32.442480 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:32.442480 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:32.442480 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:32.445441 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 323/670] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:32.446606 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde15dbd-0e75-450b-93a3-94eb8b82f6f9 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:32.492835 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:32.506001 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8052bd-e638-45bb-90b4-9155d373a7db None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:32.549450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2beaf63-3596-4def-9a05-bd1764284795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:32.550463 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2beaf63-3596-4def-9a05-bd1764284795 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:32.551014 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2beaf63-3596-4def-9a05-bd1764284795 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:32.551519 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2beaf63-3596-4def-9a05-bd1764284795 None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:32.552406 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2beaf63-3596-4def-9a05-bd1764284795 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:32.552924 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2beaf63-3596-4def-9a05-bd1764284795 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:32.554553 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2beaf63-3596-4def-9a05-bd1764284795 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:32.571468 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 348/671] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:32.572663 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2beaf63-3596-4def-9a05-bd1764284795 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:32.589684 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde15dbd-0e75-450b-93a3-94eb8b82f6f9 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:32.591898 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:32.636016 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a9997-08df-47d8-a0d5-1e8bc9f71972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:32.637078 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a9997-08df-47d8-a0d5-1e8bc9f71972 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:32.637851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a9997-08df-47d8-a0d5-1e8bc9f71972 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:32.638231 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a9997-08df-47d8-a0d5-1e8bc9f71972 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:32.650646 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68a9997-08df-47d8-a0d5-1e8bc9f71972 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:32.652350 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68a9997-08df-47d8-a0d5-1e8bc9f71972 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:32.668388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-d68a9997-08df-47d8-a0d5-1e8bc9f71972 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:32.668388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:32.668388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:32.668388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:32.668388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:32.668388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:32.668388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:32.668388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:32.668388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:32.668388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:32.668388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:32.668388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:32.668388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:32.668388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:32.668388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:32.669550 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:32.669550 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:32.669550 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:32.669550 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:32.669550 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:32.669550 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:32.669550 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:32.669550 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:32.669550 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:32.669550 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:32.669550 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:32.669550 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:32.669550 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:32.669550 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:32.669550 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:32.669550 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:32.669550 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:32.670649 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:32.670649 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:32.670649 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:32.670649 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:32.672616 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 324/672] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:32.673643 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a9997-08df-47d8-a0d5-1e8bc9f71972 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:32.692994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2beaf63-3596-4def-9a05-bd1764284795 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:32.737832 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07aff88-2c89-4ab6-aa12-f2b36232fae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:32.738890 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07aff88-2c89-4ab6-aa12-f2b36232fae9 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:32.739313 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07aff88-2c89-4ab6-aa12-f2b36232fae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:32.739740 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07aff88-2c89-4ab6-aa12-f2b36232fae9 None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:32.740611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07aff88-2c89-4ab6-aa12-f2b36232fae9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:32.741051 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07aff88-2c89-4ab6-aa12-f2b36232fae9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:32.742639 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07aff88-2c89-4ab6-aa12-f2b36232fae9 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:32.751154 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:32.758184 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 349/673] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:32.759312 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07aff88-2c89-4ab6-aa12-f2b36232fae9 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:32.792695 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a9997-08df-47d8-a0d5-1e8bc9f71972 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:32.830145 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effea681-a729-471f-ac25-20fda8f56b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:32.830782 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effea681-a729-471f-ac25-20fda8f56b22 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:32.831056 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effea681-a729-471f-ac25-20fda8f56b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:32.831326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effea681-a729-471f-ac25-20fda8f56b22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:32.838991 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effea681-a729-471f-ac25-20fda8f56b22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:32.840440 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effea681-a729-471f-ac25-20fda8f56b22 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:32.852099 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-effea681-a729-471f-ac25-20fda8f56b22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:32.852099 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:32.852099 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:32.852099 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:32.852099 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:32.852099 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:32.852099 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:32.852099 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:32.852099 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:32.852099 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:32.852099 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:32.852099 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:32.852099 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 14:07:32.852099 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 14:07:32.852099 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 14:07:32.853204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 14:07:32.853204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:32.853204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:32.853204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:32.853204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:32.853204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:32.853204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:32.853204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:32.853204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:32.853204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:32.853204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:32.853204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:32.853204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:32.853204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:32.853204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:32.853204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:32.853204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:32.858851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 14:07:32.858851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 14:07:32.858851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:07:32.858851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:07:32.858851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:07:32.858851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:07:32.858851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:32.858851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:32.858851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 325/674] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:07:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:32.858851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd0218-b202-403c-87fc-0ee08da786d5 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:32.858851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd0218-b202-403c-87fc-0ee08da786d5 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:32.859462 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd0218-b202-403c-87fc-0ee08da786d5 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:32.859462 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 326/675] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:32.859462 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd0218-b202-403c-87fc-0ee08da786d5 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:32.874520 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:32.881828 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07aff88-2c89-4ab6-aa12-f2b36232fae9 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:32.918062 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd728d15-4798-401f-bb27-14a2bf597801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:32.918761 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd728d15-4798-401f-bb27-14a2bf597801 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:32.919349 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd728d15-4798-401f-bb27-14a2bf597801 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:32.919489 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd728d15-4798-401f-bb27-14a2bf597801 None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:32.920224 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd728d15-4798-401f-bb27-14a2bf597801 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:32.920647 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd728d15-4798-401f-bb27-14a2bf597801 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:32.922049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd728d15-4798-401f-bb27-14a2bf597801 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:32.933960 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 350/676] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:32.934815 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd728d15-4798-401f-bb27-14a2bf597801 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:32.974297 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd0218-b202-403c-87fc-0ee08da786d5 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:33.000018 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:33.009325 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a19335-e99b-4bd7-9ce9-7cb74a4d3487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:33.009943 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a19335-e99b-4bd7-9ce9-7cb74a4d3487 None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:33.010220 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a19335-e99b-4bd7-9ce9-7cb74a4d3487 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:33.010490 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a19335-e99b-4bd7-9ce9-7cb74a4d3487 None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:33.011163 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a19335-e99b-4bd7-9ce9-7cb74a4d3487 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:33.012604 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a19335-e99b-4bd7-9ce9-7cb74a4d3487 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:33.043761 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd728d15-4798-401f-bb27-14a2bf597801 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:33.077047 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20951e4-8b9c-45e4-af1d-b0a369b522ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:33.077665 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20951e4-8b9c-45e4-af1d-b0a369b522ba None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:33.077941 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20951e4-8b9c-45e4-af1d-b0a369b522ba None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:33.078210 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20951e4-8b9c-45e4-af1d-b0a369b522ba None admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:33.078954 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20951e4-8b9c-45e4-af1d-b0a369b522ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:33.079258 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20951e4-8b9c-45e4-af1d-b0a369b522ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:33.080712 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20951e4-8b9c-45e4-af1d-b0a369b522ba None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:33.094121 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 351/677] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Aug 5 14:07:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:33.095653 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acf695-5797-4ebb-b941-ee545851e5fa None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:33.096076 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acf695-5797-4ebb-b941-ee545851e5fa None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:33.102093 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acf695-5797-4ebb-b941-ee545851e5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:33.102244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 327/678] 10.222.0.26 () {60 vars in 1132 bytes} [Fri Aug 5 14:07:32 2022] POST /identity/v3/users => generated 254 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:07:33.102244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a19335-e99b-4bd7-9ce9-7cb74a4d3487 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:33.122089 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-83acf695-5797-4ebb-b941-ee545851e5fa None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:33.122912 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-83acf695-5797-4ebb-b941-ee545851e5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:33.208953 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 352/679] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:07:33.209916 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acf695-5797-4ebb-b941-ee545851e5fa None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:33.214254 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:33.254349 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a19335-e99b-4bd7-9ce9-7cb74a4d3487 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:33.280304 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:33.298650 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea88ba78-d2be-4ea0-a3fc-e7d3ffda7fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:33.308821 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88ba78-d2be-4ea0-a3fc-e7d3ffda7fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:33.308821 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88ba78-d2be-4ea0-a3fc-e7d3ffda7fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:33.308821 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88ba78-d2be-4ea0-a3fc-e7d3ffda7fd7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:33.308821 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea88ba78-d2be-4ea0-a3fc-e7d3ffda7fd7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:33.311325 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea88ba78-d2be-4ea0-a3fc-e7d3ffda7fd7 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:33.325897 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-ea88ba78-d2be-4ea0-a3fc-e7d3ffda7fd7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:33.325897 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:33.325897 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:33.325897 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:33.325897 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:33.325897 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:33.325897 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:33.325897 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:33.325897 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:33.325897 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:33.325897 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:33.325897 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:33.325897 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 14:07:33.325897 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 14:07:33.325897 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 14:07:33.326939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 14:07:33.326939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:33.326939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:33.326939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:33.326939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:33.326939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:33.326939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:33.326939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:33.326939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:33.326939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:33.326939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:33.326939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:33.326939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:33.326939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:33.326939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:33.326939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:33.326939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:33.328087 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 14:07:33.328087 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 14:07:33.328087 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:07:33.328087 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:07:33.328087 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:07:33.328087 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:07:33.328087 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:33.328087 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:33.331433 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 328/680] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:07:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:33.333198 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea88ba78-d2be-4ea0-a3fc-e7d3ffda7fd7 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:33.347029 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acf695-5797-4ebb-b941-ee545851e5fa None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:33.390637 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13eafd-06a8-4920-ac2a-9b602dfed42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:33.391629 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13eafd-06a8-4920-ac2a-9b602dfed42e None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:33.392157 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13eafd-06a8-4920-ac2a-9b602dfed42e None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:33.392646 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13eafd-06a8-4920-ac2a-9b602dfed42e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:33.393607 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb13eafd-06a8-4920-ac2a-9b602dfed42e None admin] RBAC: Adding query filter params (role.id=ec18a2850fd540a39d4a935348bb4474, scope.project.id=319973e011c54b4ca1f2198c8a5e135b, user.id=88d6a1bae9834e2e91d4ab3a4c00959b) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:33.394095 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb13eafd-06a8-4920-ac2a-9b602dfed42e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:33.396312 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb13eafd-06a8-4920-ac2a-9b602dfed42e None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:33.403850 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 353/681] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:07:33 2022] GET /identity/v3/role_assignments?user.id=88d6a1bae9834e2e91d4ab3a4c00959b&scope.project.id=319973e011c54b4ca1f2198c8a5e135b&role.id=ec18a2850fd540a39d4a935348bb4474 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:33.404860 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13eafd-06a8-4920-ac2a-9b602dfed42e None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:33.449798 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea88ba78-d2be-4ea0-a3fc-e7d3ffda7fd7 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:33.481971 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:33.487788 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e6176-5714-4145-9ff6-a9db5beff6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:33.488688 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e6176-5714-4145-9ff6-a9db5beff6af None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:33.489075 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e6176-5714-4145-9ff6-a9db5beff6af None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:33.491373 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e6176-5714-4145-9ff6-a9db5beff6af None admin] PATH_INFO: `/v3/roles/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:33.501895 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15e6176-5714-4145-9ff6-a9db5beff6af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:07:33.507916 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-c15e6176-5714-4145-9ff6-a9db5beff6af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:07:33.507916 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:33.507916 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:33.507916 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:33.507916 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:33.507916 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:33.507916 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:33.507916 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:33.507916 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:33.507916 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:33.507916 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:33.507916 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:33.507916 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:07:33.507916 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:07:33.507916 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:07:33.507916 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:07:33.509063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:07:33.509063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:07:33.509063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:07:33.509063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:07:33.509063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:33.509063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:33.509063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:33.509063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:33.509063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:33.509063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:33.509063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:33.509063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:33.509063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:33.509063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:33.509063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:33.509063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:33.509916 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:33.509916 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:33.509916 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:33.509916 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:33.509916 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:07:33.509916 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:07:33.509916 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:07:33.509916 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:07:33.509916 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:07:33.509916 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:07:33.509916 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:07:33.509916 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:33.510588 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 329/682] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Aug 5 14:07:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:33.525035 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e6176-5714-4145-9ff6-a9db5beff6af None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:33.637504 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e6176-5714-4145-9ff6-a9db5beff6af None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:33.646536 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13eafd-06a8-4920-ac2a-9b602dfed42e None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:33.673348 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:33.676474 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331eda0b-a958-4cce-ab58-af7a7aaa9215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:33.677111 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331eda0b-a958-4cce-ab58-af7a7aaa9215 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:33.677392 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331eda0b-a958-4cce-ab58-af7a7aaa9215 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:33.677669 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331eda0b-a958-4cce-ab58-af7a7aaa9215 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:33.678381 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331eda0b-a958-4cce-ab58-af7a7aaa9215 None admin] RBAC: Adding query filter params (name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:33.678692 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331eda0b-a958-4cce-ab58-af7a7aaa9215 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:33.680201 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331eda0b-a958-4cce-ab58-af7a7aaa9215 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:33.689442 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 330/683] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Aug 5 14:07:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:33.695461 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331eda0b-a958-4cce-ab58-af7a7aaa9215 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:33.709441 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af899655-f8a3-4284-8d0b-26a3643ca2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:33.710464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af899655-f8a3-4284-8d0b-26a3643ca2e9 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:33.710948 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af899655-f8a3-4284-8d0b-26a3643ca2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:33.711432 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af899655-f8a3-4284-8d0b-26a3643ca2e9 None admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:33.712517 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af899655-f8a3-4284-8d0b-26a3643ca2e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:33.713079 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af899655-f8a3-4284-8d0b-26a3643ca2e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:33.714769 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af899655-f8a3-4284-8d0b-26a3643ca2e9 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:33.733602 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 354/684] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Aug 5 14:07:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:33.739902 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af899655-f8a3-4284-8d0b-26a3643ca2e9 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:33.768742 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:33.825843 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331eda0b-a958-4cce-ab58-af7a7aaa9215 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:33.867986 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8847c42-a83e-4b8a-af47-e72d974972fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:33.869290 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8847c42-a83e-4b8a-af47-e72d974972fd None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:33.870191 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8847c42-a83e-4b8a-af47-e72d974972fd None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:33.871189 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8847c42-a83e-4b8a-af47-e72d974972fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:33.876251 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af899655-f8a3-4284-8d0b-26a3643ca2e9 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:33.886124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8847c42-a83e-4b8a-af47-e72d974972fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:33.887984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8847c42-a83e-4b8a-af47-e72d974972fd None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:33.908970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-e8847c42-a83e-4b8a-af47-e72d974972fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:33.908970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:33.908970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:33.908970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:33.908970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:33.908970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:33.908970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:33.908970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:33.908970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:33.908970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:33.908970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:33.908970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:33.908970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:33.908970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:33.908970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:33.910559 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:33.910559 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:33.910559 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:33.910559 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:33.910559 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:33.910559 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:33.910559 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:33.910559 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:33.910559 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:33.910559 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:33.910559 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:33.910559 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:33.910559 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:33.910559 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:33.910559 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:33.910559 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:33.910559 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:33.911490 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:33.911490 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:33.911490 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:33.911490 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:33.914269 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 331/685] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:33.921454 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8847c42-a83e-4b8a-af47-e72d974972fd None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:33.922487 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556bf2a3-80b5-441e-80de-c8ebad2dbd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:33.923173 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bf2a3-80b5-441e-80de-c8ebad2dbd19 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:33.923449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bf2a3-80b5-441e-80de-c8ebad2dbd19 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:33.923719 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bf2a3-80b5-441e-80de-c8ebad2dbd19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:33.924569 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556bf2a3-80b5-441e-80de-c8ebad2dbd19 None admin] RBAC: Adding query filter params (role.id=ec18a2850fd540a39d4a935348bb4474, scope.project.id=319973e011c54b4ca1f2198c8a5e135b, user.id=695e37560f8f495e987d1bd17acc2680) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:33.924880 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556bf2a3-80b5-441e-80de-c8ebad2dbd19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:33.926950 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556bf2a3-80b5-441e-80de-c8ebad2dbd19 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:33.933958 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 355/686] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:07:33 2022] GET /identity/v3/role_assignments?user.id=695e37560f8f495e987d1bd17acc2680&scope.project.id=319973e011c54b4ca1f2198c8a5e135b&role.id=ec18a2850fd540a39d4a935348bb4474 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:34.072013 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8847c42-a83e-4b8a-af47-e72d974972fd None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:34.084933 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:34.101330 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4512f-5b13-48ce-b47f-9d0b1db7ae47 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:34.101631 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4512f-5b13-48ce-b47f-9d0b1db7ae47 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:34.106820 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4512f-5b13-48ce-b47f-9d0b1db7ae47 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:34.106820 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 356/687] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:34.110497 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58a815-8aa6-458d-b29b-3cfcc9d53c12 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:34.111015 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58a815-8aa6-458d-b29b-3cfcc9d53c12 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:34.111478 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58a815-8aa6-458d-b29b-3cfcc9d53c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:34.117315 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59ef7e8-e3b0-4b1c-be00-365223f0b95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:34.117960 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ef7e8-e3b0-4b1c-be00-365223f0b95d None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:34.118291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ef7e8-e3b0-4b1c-be00-365223f0b95d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:34.118563 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ef7e8-e3b0-4b1c-be00-365223f0b95d None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:34.119503 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59ef7e8-e3b0-4b1c-be00-365223f0b95d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:34.119827 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59ef7e8-e3b0-4b1c-be00-365223f0b95d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:34.122449 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59ef7e8-e3b0-4b1c-be00-365223f0b95d None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:34.133753 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-0f58a815-8aa6-458d-b29b-3cfcc9d53c12 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:34.135288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-0f58a815-8aa6-458d-b29b-3cfcc9d53c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:34.147865 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 332/688] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:34.156075 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59ef7e8-e3b0-4b1c-be00-365223f0b95d None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:34.196668 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:34.245592 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 357/689] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:07:34.281064 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58a815-8aa6-458d-b29b-3cfcc9d53c12 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:34.310211 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59ef7e8-e3b0-4b1c-be00-365223f0b95d None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:34.358202 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ffc764-18ef-4afc-934c-391dc3ed7420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:34.359201 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffc764-18ef-4afc-934c-391dc3ed7420 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:34.359486 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffc764-18ef-4afc-934c-391dc3ed7420 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:34.359807 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffc764-18ef-4afc-934c-391dc3ed7420 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:34.370771 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ffc764-18ef-4afc-934c-391dc3ed7420 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:34.373135 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ffc764-18ef-4afc-934c-391dc3ed7420 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:34.389724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-f3ffc764-18ef-4afc-934c-391dc3ed7420 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 14:07:34.389724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:34.389724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:34.389724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:34.389724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:34.389724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:34.389724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:34.389724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:34.389724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:34.389724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:34.389724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:34.389724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:34.389724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 14:07:34.389724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 14:07:34.389724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 14:07:34.389724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 14:07:34.390770 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:34.390770 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:34.390770 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:07:34.390770 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:34.390770 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:07:34.390770 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:34.390770 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:34.390770 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:34.390770 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:34.390770 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:34.390770 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:34.390770 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:34.390770 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:34.390770 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:34.390770 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:34.390770 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:34.390770 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:34.390770 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:34.391752 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:34.391752 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:34.391752 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 14:07:34.391752 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 14:07:34.391752 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 14:07:34.391752 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 14:07:34.391752 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 14:07:34.391752 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 14:07:34.391752 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 14:07:34.391752 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:34.391752 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 333/690] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Aug 5 14:07:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:34.398214 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ffc764-18ef-4afc-934c-391dc3ed7420 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:34.418830 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58a815-8aa6-458d-b29b-3cfcc9d53c12 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:34.440455 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:34.499459 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc6f58-793e-49f6-bd64-678192d5367a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:34.500107 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc6f58-793e-49f6-bd64-678192d5367a None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:34.500385 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc6f58-793e-49f6-bd64-678192d5367a None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:34.500682 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc6f58-793e-49f6-bd64-678192d5367a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:34.512185 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fc6f58-793e-49f6-bd64-678192d5367a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:07:34.517526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-02fc6f58-793e-49f6-bd64-678192d5367a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:07:34.517526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:34.517526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:34.517526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:34.517526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:34.517526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:34.517526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:34.517526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:34.517526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:34.517526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:34.517526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:34.517526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:34.517526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:07:34.517526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:07:34.517526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:07:34.517526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:07:34.518464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:07:34.518464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:07:34.518464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:07:34.518464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:07:34.518464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:34.518464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:34.518464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:34.518464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:34.518464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:34.518464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:34.518464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:34.518464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:34.518464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:34.518464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:34.518464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:34.518464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:34.519868 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:34.519868 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:34.519868 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:34.519868 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:34.519868 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:07:34.519868 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:07:34.519868 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:07:34.519868 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:07:34.519868 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:07:34.519868 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:07:34.519868 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:07:34.519868 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:34.519868 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 358/691] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Aug 5 14:07:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:34.527261 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc6f58-793e-49f6-bd64-678192d5367a None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:34.588658 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ffc764-18ef-4afc-934c-391dc3ed7420 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:34.645419 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc56f60-11ed-4eac-9638-6fd07f7247c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:34.646089 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc56f60-11ed-4eac-9638-6fd07f7247c9 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:34.648447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc56f60-11ed-4eac-9638-6fd07f7247c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:34.648447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc56f60-11ed-4eac-9638-6fd07f7247c9 None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:34.648447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc56f60-11ed-4eac-9638-6fd07f7247c9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:34.648447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc56f60-11ed-4eac-9638-6fd07f7247c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:34.649316 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc56f60-11ed-4eac-9638-6fd07f7247c9 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:34.683267 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 334/692] 10.222.0.26 () {56 vars in 1135 bytes} [Fri Aug 5 14:07:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:34.688060 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc56f60-11ed-4eac-9638-6fd07f7247c9 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:34.775150 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:34.776999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:34.795011 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc56f60-11ed-4eac-9638-6fd07f7247c9 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:34.836447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb6e2a-dbbd-45b9-a565-84279ad21a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:34.836447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb6e2a-dbbd-45b9-a565-84279ad21a69 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:34.836447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb6e2a-dbbd-45b9-a565-84279ad21a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:34.836447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb6e2a-dbbd-45b9-a565-84279ad21a69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:34.904911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc6f58-793e-49f6-bd64-678192d5367a None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:34.914895 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcb6e2a-dbbd-45b9-a565-84279ad21a69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:34.916432 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcb6e2a-dbbd-45b9-a565-84279ad21a69 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:34.968040 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da0e840-5028-4e95-92cc-ce62a6487506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:34.969070 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da0e840-5028-4e95-92cc-ce62a6487506 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:34.969662 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da0e840-5028-4e95-92cc-ce62a6487506 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:34.970214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da0e840-5028-4e95-92cc-ce62a6487506 None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:34.971246 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da0e840-5028-4e95-92cc-ce62a6487506 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:34.971754 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da0e840-5028-4e95-92cc-ce62a6487506 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:34.973834 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da0e840-5028-4e95-92cc-ce62a6487506 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:34.993539 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 359/693] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:07:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 468 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:35.000145 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da0e840-5028-4e95-92cc-ce62a6487506 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:35.022723 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:35.034113 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 335/694] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:07:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 14:07:35.040696 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb6e2a-dbbd-45b9-a565-84279ad21a69 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:35.192047 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb6e2a-dbbd-45b9-a565-84279ad21a69 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:35.197958 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:35.219946 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da0e840-5028-4e95-92cc-ce62a6487506 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:35.258677 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f024c84-1f26-4819-925e-4466ec45ff77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:35.259321 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f024c84-1f26-4819-925e-4466ec45ff77 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:35.259594 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f024c84-1f26-4819-925e-4466ec45ff77 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:35.259865 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f024c84-1f26-4819-925e-4466ec45ff77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:35.276080 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f024c84-1f26-4819-925e-4466ec45ff77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:35.277544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f024c84-1f26-4819-925e-4466ec45ff77 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:35.290964 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406bb96-52da-4f35-bb29-3233bd908499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:35.291612 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406bb96-52da-4f35-bb29-3233bd908499 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:35.291895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406bb96-52da-4f35-bb29-3233bd908499 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:35.292160 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406bb96-52da-4f35-bb29-3233bd908499 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:35.305414 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-7f024c84-1f26-4819-925e-4466ec45ff77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:35.305414 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:35.305414 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:35.305414 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:35.305414 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:35.305414 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:35.305414 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:35.305414 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:35.305414 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:35.305414 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:35.305414 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:35.305414 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:35.305414 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:35.305414 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:35.305414 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:35.306364 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:35.306364 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:35.306364 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:35.306364 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:35.306364 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:35.306364 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:35.306364 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:35.306364 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:35.306364 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:35.306364 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:35.306364 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:35.306364 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:35.306364 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:35.306364 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:35.306364 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:35.306364 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:35.306364 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:35.307266 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:35.307266 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:35.307266 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:35.307266 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:35.312383 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b406bb96-52da-4f35-bb29-3233bd908499 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:35.312565 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 336/695] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:07:35.312655 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b406bb96-52da-4f35-bb29-3233bd908499 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:35.313977 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f024c84-1f26-4819-925e-4466ec45ff77 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:35.340478 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-b406bb96-52da-4f35-bb29-3233bd908499 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:35.340478 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:35.340478 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:35.340478 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:35.340478 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:35.340478 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:35.340478 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:35.340478 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:35.340478 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:35.340478 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:35.340478 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:35.340478 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:35.340478 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:35.340478 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:35.340478 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:35.342568 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:35.342568 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:35.342568 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:35.342568 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:35.342568 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:35.342568 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:35.342568 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:35.342568 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:35.342568 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:35.342568 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:35.342568 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:35.342568 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:35.342568 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:35.342568 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:35.342568 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:35.342568 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:35.342568 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:35.344687 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:35.344687 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:35.344687 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:35.344687 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:35.344687 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 360/696] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:07:35.349363 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406bb96-52da-4f35-bb29-3233bd908499 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:35.392774 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:35.480631 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f024c84-1f26-4819-925e-4466ec45ff77 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:35.518299 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae61e28-74b1-4bff-981b-3667c7ff736b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:35.518959 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae61e28-74b1-4bff-981b-3667c7ff736b None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:35.519762 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae61e28-74b1-4bff-981b-3667c7ff736b None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:35.520036 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae61e28-74b1-4bff-981b-3667c7ff736b None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:35.520867 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae61e28-74b1-4bff-981b-3667c7ff736b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:35.521173 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae61e28-74b1-4bff-981b-3667c7ff736b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:35.522840 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae61e28-74b1-4bff-981b-3667c7ff736b None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:35.537647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 337/697] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:35.538975 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203414a-2ac9-4fac-a1b0-78f3c7443391 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:35.539261 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203414a-2ac9-4fac-a1b0-78f3c7443391 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:35.539533 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203414a-2ac9-4fac-a1b0-78f3c7443391 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:35.544743 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 338/698] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:35.544743 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0203414a-2ac9-4fac-a1b0-78f3c7443391 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:35.553151 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406bb96-52da-4f35-bb29-3233bd908499 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:35.597489 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dbdd45-5fea-4bf9-8416-6d06e3c25f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:35.598980 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbdd45-5fea-4bf9-8416-6d06e3c25f79 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:35.599486 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbdd45-5fea-4bf9-8416-6d06e3c25f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:35.600235 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbdd45-5fea-4bf9-8416-6d06e3c25f79 None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:35.601330 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dbdd45-5fea-4bf9-8416-6d06e3c25f79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:35.602193 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dbdd45-5fea-4bf9-8416-6d06e3c25f79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:35.604230 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dbdd45-5fea-4bf9-8416-6d06e3c25f79 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:35.617434 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:35.624745 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 361/699] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:35.624745 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78309d07-f86e-4cda-982d-2a544a5e89bb None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:35.624745 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78309d07-f86e-4cda-982d-2a544a5e89bb None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:35.624745 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78309d07-f86e-4cda-982d-2a544a5e89bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:35.649141 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-78309d07-f86e-4cda-982d-2a544a5e89bb None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:35.650338 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-78309d07-f86e-4cda-982d-2a544a5e89bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:35.660101 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:35.667569 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0203414a-2ac9-4fac-a1b0-78f3c7443391 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:35.702195 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed941fc-945b-4455-9d56-36a6f0491d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:35.702842 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed941fc-945b-4455-9d56-36a6f0491d31 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:35.703123 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed941fc-945b-4455-9d56-36a6f0491d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:35.703399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed941fc-945b-4455-9d56-36a6f0491d31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:35.711607 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed941fc-945b-4455-9d56-36a6f0491d31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:35.713118 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed941fc-945b-4455-9d56-36a6f0491d31 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:35.726189 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-0ed941fc-945b-4455-9d56-36a6f0491d31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:35.726189 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:35.726189 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:35.726189 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:35.726189 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:35.726189 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:35.726189 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:35.726189 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:35.726189 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:35.726189 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:35.726189 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:35.726189 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:35.726189 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 14:07:35.726189 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 14:07:35.726189 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 14:07:35.727346 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 14:07:35.727346 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:35.727346 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:35.727346 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:35.727346 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:35.727346 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:35.727346 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:35.727346 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:35.727346 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:35.727346 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:35.727346 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:35.727346 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:35.727346 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:35.727346 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:35.727346 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:35.727346 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:35.727346 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:35.728536 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 14:07:35.728536 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 14:07:35.728536 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:07:35.728536 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:07:35.728536 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:07:35.728536 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:07:35.728536 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:35.728536 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:35.732847 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 339/700] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:07:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:35.733838 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed941fc-945b-4455-9d56-36a6f0491d31 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:35.744688 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 362/701] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:07:35.746172 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78309d07-f86e-4cda-982d-2a544a5e89bb None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:35.871473 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78309d07-f86e-4cda-982d-2a544a5e89bb None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:35.876009 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed941fc-945b-4455-9d56-36a6f0491d31 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:35.907102 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:35.915740 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e34b2d-fa58-4ece-aa7b-dde127c139ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:35.916358 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e34b2d-fa58-4ece-aa7b-dde127c139ac None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:35.916676 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e34b2d-fa58-4ece-aa7b-dde127c139ac None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:35.917093 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e34b2d-fa58-4ece-aa7b-dde127c139ac None admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:35.917846 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e34b2d-fa58-4ece-aa7b-dde127c139ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:35.918447 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e34b2d-fa58-4ece-aa7b-dde127c139ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:35.920351 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e34b2d-fa58-4ece-aa7b-dde127c139ac None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:35.928970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22e010-014a-4ad4-a46f-ac344defa7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:35.928970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22e010-014a-4ad4-a46f-ac344defa7c6 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:35.928970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22e010-014a-4ad4-a46f-ac344defa7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:35.928970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22e010-014a-4ad4-a46f-ac344defa7c6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:35.932767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 363/702] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Aug 5 14:07:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:35.937257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e34b2d-fa58-4ece-aa7b-dde127c139ac None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:35.939341 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22e010-014a-4ad4-a46f-ac344defa7c6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:35.941396 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22e010-014a-4ad4-a46f-ac344defa7c6 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:35.953830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-da22e010-014a-4ad4-a46f-ac344defa7c6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 14:07:35.953830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:35.953830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:35.953830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:35.953830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:35.953830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:35.953830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:35.953830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:35.953830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:35.953830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:35.953830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:35.953830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:35.953830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 14:07:35.953830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 14:07:35.953830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 14:07:35.953830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 14:07:35.954817 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:35.954817 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:35.954817 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:07:35.954817 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:35.954817 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:07:35.954817 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:35.954817 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:35.954817 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:35.954817 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:35.954817 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:35.954817 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:35.954817 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:35.954817 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:35.954817 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:35.954817 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:35.954817 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:35.954817 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:35.954817 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:35.955828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:35.955828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:35.955828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 14:07:35.955828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 14:07:35.955828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 14:07:35.955828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 14:07:35.955828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 14:07:35.955828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 14:07:35.955828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 14:07:35.955828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:35.955828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 340/703] 10.222.0.26 () {56 vars in 1093 bytes} [Fri Aug 5 14:07:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:35.955828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22e010-014a-4ad4-a46f-ac344defa7c6 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:35.973419 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:36.045659 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e34b2d-fa58-4ece-aa7b-dde127c139ac None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:36.076270 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22e010-014a-4ad4-a46f-ac344defa7c6 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:36.082039 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb10f1-321a-4f38-b256-b9b0ca759e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:36.082963 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb10f1-321a-4f38-b256-b9b0ca759e0c None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:36.083341 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb10f1-321a-4f38-b256-b9b0ca759e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:36.083711 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb10f1-321a-4f38-b256-b9b0ca759e0c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:36.095023 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eb10f1-321a-4f38-b256-b9b0ca759e0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:07:36.101264 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-40eb10f1-321a-4f38-b256-b9b0ca759e0c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:07:36.101264 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:36.101264 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:36.101264 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:36.101264 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:36.101264 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:36.101264 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:36.101264 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:36.101264 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:36.101264 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:36.101264 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:36.101264 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:36.101264 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:07:36.101264 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:07:36.101264 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:07:36.101264 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:07:36.102382 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:07:36.102382 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:07:36.102382 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:07:36.102382 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:07:36.102382 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:36.102382 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:36.102382 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:36.102382 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:36.102382 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:36.102382 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:36.102382 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:36.102382 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:36.102382 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:36.102382 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:36.102382 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:36.102382 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:36.103422 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:36.103422 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:36.103422 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:36.103422 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:36.103422 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:07:36.103422 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:07:36.103422 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:07:36.103422 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:07:36.103422 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:07:36.103422 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:07:36.103422 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:07:36.103422 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:36.104758 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 364/704] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Aug 5 14:07:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:36.105975 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75dec8-c77b-4f44-a61b-9c873a6fa883 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:36.106261 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75dec8-c77b-4f44-a61b-9c873a6fa883 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:36.106526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75dec8-c77b-4f44-a61b-9c873a6fa883 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:36.107923 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 365/705] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:36.112339 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75dec8-c77b-4f44-a61b-9c873a6fa883 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:36.122121 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde807f5-2bc4-4d46-af8b-b6337df1d289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:36.122885 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde807f5-2bc4-4d46-af8b-b6337df1d289 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:36.123260 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde807f5-2bc4-4d46-af8b-b6337df1d289 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:36.123628 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde807f5-2bc4-4d46-af8b-b6337df1d289 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:36.124551 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde807f5-2bc4-4d46-af8b-b6337df1d289 None admin] RBAC: Adding query filter params (role.id=ec18a2850fd540a39d4a935348bb4474, scope.project.id=319973e011c54b4ca1f2198c8a5e135b, user.id=dddd7eea67f34f0cbb5f6a1cd85f1fc5) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:36.124930 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde807f5-2bc4-4d46-af8b-b6337df1d289 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:36.126406 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde807f5-2bc4-4d46-af8b-b6337df1d289 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:36.140460 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 341/706] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:07:35 2022] GET /identity/v3/role_assignments?user.id=dddd7eea67f34f0cbb5f6a1cd85f1fc5&scope.project.id=319973e011c54b4ca1f2198c8a5e135b&role.id=ec18a2850fd540a39d4a935348bb4474 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:36.140460 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde807f5-2bc4-4d46-af8b-b6337df1d289 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:36.188489 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:36.238606 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:36.265130 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde807f5-2bc4-4d46-af8b-b6337df1d289 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:36.267467 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75dec8-c77b-4f44-a61b-9c873a6fa883 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:36.311004 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f70a7f-0a4c-46c3-9e88-16e487ecf86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:36.311902 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f70a7f-0a4c-46c3-9e88-16e487ecf86c None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:36.312345 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f70a7f-0a4c-46c3-9e88-16e487ecf86c None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:36.312776 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f70a7f-0a4c-46c3-9e88-16e487ecf86c None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:36.313631 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f70a7f-0a4c-46c3-9e88-16e487ecf86c None admin] RBAC: Adding query filter params (name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:36.314067 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f70a7f-0a4c-46c3-9e88-16e487ecf86c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:36.315567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f70a7f-0a4c-46c3-9e88-16e487ecf86c None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:36.325149 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74058179-33c0-4f11-9fca-301c19ebefc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:36.325871 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74058179-33c0-4f11-9fca-301c19ebefc3 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:36.326221 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74058179-33c0-4f11-9fca-301c19ebefc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:36.326556 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74058179-33c0-4f11-9fca-301c19ebefc3 None admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:36.327433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74058179-33c0-4f11-9fca-301c19ebefc3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:36.328330 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74058179-33c0-4f11-9fca-301c19ebefc3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:36.330045 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74058179-33c0-4f11-9fca-301c19ebefc3 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:36.333359 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 342/707] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Aug 5 14:07:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:36.335711 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71d9a8-6ea7-4ea3-8504-86d5a93e2658 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:36.336148 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71d9a8-6ea7-4ea3-8504-86d5a93e2658 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:36.336590 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71d9a8-6ea7-4ea3-8504-86d5a93e2658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:36.357533 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 366/708] 10.222.0.26 () {56 vars in 1133 bytes} [Fri Aug 5 14:07:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:36.357810 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-ff71d9a8-6ea7-4ea3-8504-86d5a93e2658 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:36.359140 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ff71d9a8-6ea7-4ea3-8504-86d5a93e2658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:36.359413 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74058179-33c0-4f11-9fca-301c19ebefc3 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:36.443211 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 343/709] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:07:36.445050 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff71d9a8-6ea7-4ea3-8504-86d5a93e2658 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:36.481175 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74058179-33c0-4f11-9fca-301c19ebefc3 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:36.493870 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:36.532975 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:36.539345 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7b026-14ed-412e-bbc2-586ccb9d6054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:36.540731 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7b026-14ed-412e-bbc2-586ccb9d6054 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:36.541213 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7b026-14ed-412e-bbc2-586ccb9d6054 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:36.541681 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7b026-14ed-412e-bbc2-586ccb9d6054 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:36.555765 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7b026-14ed-412e-bbc2-586ccb9d6054 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:36.558305 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7b026-14ed-412e-bbc2-586ccb9d6054 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:36.567859 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff71d9a8-6ea7-4ea3-8504-86d5a93e2658 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:36.578779 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-1bd7b026-14ed-412e-bbc2-586ccb9d6054 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:36.578779 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:36.578779 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:36.578779 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:36.578779 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:36.578779 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:36.578779 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:36.578779 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:36.578779 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:36.578779 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:36.578779 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:36.578779 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:36.578779 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:36.578779 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:36.578779 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:36.579766 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:36.579766 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:36.579766 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:36.579766 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:36.579766 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:36.579766 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:36.579766 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:36.579766 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:36.579766 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:36.579766 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:36.579766 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:36.579766 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:36.579766 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:36.579766 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:36.579766 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:36.579766 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:36.579766 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:36.581388 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:36.581388 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:36.581388 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:36.581388 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:36.583695 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 367/710] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:36.588731 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7b026-14ed-412e-bbc2-586ccb9d6054 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:36.623441 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eefb81c-9bf9-448e-a25b-d240c5f30566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:36.624709 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eefb81c-9bf9-448e-a25b-d240c5f30566 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:36.625189 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eefb81c-9bf9-448e-a25b-d240c5f30566 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:36.625669 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eefb81c-9bf9-448e-a25b-d240c5f30566 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:36.703229 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7b026-14ed-412e-bbc2-586ccb9d6054 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:36.710533 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eefb81c-9bf9-448e-a25b-d240c5f30566 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:36.711981 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eefb81c-9bf9-448e-a25b-d240c5f30566 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:36.739581 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd74a693-84ae-4652-9b4b-4359d5ec0c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:36.740235 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74a693-84ae-4652-9b4b-4359d5ec0c4b None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:36.740536 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74a693-84ae-4652-9b4b-4359d5ec0c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:36.740802 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74a693-84ae-4652-9b4b-4359d5ec0c4b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:36.741436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd74a693-84ae-4652-9b4b-4359d5ec0c4b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:36.742779 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd74a693-84ae-4652-9b4b-4359d5ec0c4b None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:36.754350 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-bd74a693-84ae-4652-9b4b-4359d5ec0c4b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 14:07:36.754350 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:36.754350 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 14:07:36.754350 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 14:07:36.754350 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 14:07:36.754350 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 14:07:36.754350 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 14:07:36.754350 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:07:36.754350 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 14:07:36.754350 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:36.754350 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 14:07:36.754350 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:36.754350 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:36.754350 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:36.754350 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:36.754350 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:36.755353 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:36.755353 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:36.755353 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:36.755353 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:36.755353 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:36.755353 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:36.755353 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:36.755353 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 14:07:36.755353 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 14:07:36.755353 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 14:07:36.755353 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 14:07:36.755353 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:36.755353 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:36.755353 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:36.755353 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:36.755353 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:36.761164 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:36.761164 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:36.761164 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:36.761164 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:36.761164 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:36.761164 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:36.761164 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:36.761164 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:36.761164 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:36.761164 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:36.761164 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:36.761164 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 14:07:36.761164 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:07:36.761164 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 14:07:36.761164 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:36.761164 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 368/711] 10.222.0.26 () {56 vars in 1129 bytes} [Fri Aug 5 14:07:36 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:36.762044 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd74a693-84ae-4652-9b4b-4359d5ec0c4b None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:36.786479 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:36.795617 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:36.834242 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 344/712] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:07:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 14:07:36.835581 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca4fdd-aaa0-44d4-afd7-7a46391a5ff6 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:36.835875 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca4fdd-aaa0-44d4-afd7-7a46391a5ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:36.836141 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca4fdd-aaa0-44d4-afd7-7a46391a5ff6 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:36.837595 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 345/713] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:36.838273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca4fdd-aaa0-44d4-afd7-7a46391a5ff6 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:36.877208 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd74a693-84ae-4652-9b4b-4359d5ec0c4b None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:36.912776 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba6d89-507d-4766-8f8b-606004c64aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:36.913387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba6d89-507d-4766-8f8b-606004c64aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:36.913659 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba6d89-507d-4766-8f8b-606004c64aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:36.914025 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba6d89-507d-4766-8f8b-606004c64aa7 None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:36.914590 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ba6d89-507d-4766-8f8b-606004c64aa7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:36.914886 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ba6d89-507d-4766-8f8b-606004c64aa7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:36.916252 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ba6d89-507d-4766-8f8b-606004c64aa7 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:36.928368 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 369/714] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:36.929481 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba6d89-507d-4766-8f8b-606004c64aa7 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:36.957943 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca4fdd-aaa0-44d4-afd7-7a46391a5ff6 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:36.994043 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c734b30-6d7e-475c-abb6-5d4ecdb13212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:36.994804 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c734b30-6d7e-475c-abb6-5d4ecdb13212 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:36.995184 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c734b30-6d7e-475c-abb6-5d4ecdb13212 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:36.995561 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c734b30-6d7e-475c-abb6-5d4ecdb13212 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:36.996597 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c734b30-6d7e-475c-abb6-5d4ecdb13212 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:36.996987 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c734b30-6d7e-475c-abb6-5d4ecdb13212 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:36.998444 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c734b30-6d7e-475c-abb6-5d4ecdb13212 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:37.003993 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 346/715] 10.222.0.26 () {56 vars in 1124 bytes} [Fri Aug 5 14:07:36 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:37.005170 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33238cba-d99a-4be7-a5ba-12e28ff19eff None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:37.005460 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33238cba-d99a-4be7-a5ba-12e28ff19eff None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:37.005735 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33238cba-d99a-4be7-a5ba-12e28ff19eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:37.024657 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-33238cba-d99a-4be7-a5ba-12e28ff19eff None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:37.025860 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-33238cba-d99a-4be7-a5ba-12e28ff19eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:37.026540 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:37.044013 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba6d89-507d-4766-8f8b-606004c64aa7 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:37.054715 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:37.079711 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b36e21-d122-43b4-91a7-14c654305b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:37.080329 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b36e21-d122-43b4-91a7-14c654305b90 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:37.080631 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b36e21-d122-43b4-91a7-14c654305b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:37.080904 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b36e21-d122-43b4-91a7-14c654305b90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:37.091445 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b36e21-d122-43b4-91a7-14c654305b90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:37.092840 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b36e21-d122-43b4-91a7-14c654305b90 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:37.100490 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 347/716] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:07:37.100490 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33238cba-d99a-4be7-a5ba-12e28ff19eff None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:37.117449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-a7b36e21-d122-43b4-91a7-14c654305b90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:37.117449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:37.117449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:37.117449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:37.117449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:37.117449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:37.117449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:37.117449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:37.117449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:37.117449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:37.117449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:37.117449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:37.117449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:37.117449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:37.117449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:37.118583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:37.118583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:37.118583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:37.118583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:37.118583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:37.118583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:37.118583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:37.118583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:37.118583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:37.118583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:37.118583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:37.118583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:37.118583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:37.118583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:37.118583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:37.118583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:37.118583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:37.119747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:37.119747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:37.119747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:37.119747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:37.120958 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 370/717] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:37.122192 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b36e21-d122-43b4-91a7-14c654305b90 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:37.202550 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33238cba-d99a-4be7-a5ba-12e28ff19eff None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:37.235611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b36e21-d122-43b4-91a7-14c654305b90 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:37.235780 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd609a-5017-4260-ac18-951fae3d20f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:37.236281 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd609a-5017-4260-ac18-951fae3d20f6 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:37.236660 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd609a-5017-4260-ac18-951fae3d20f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:37.236930 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd609a-5017-4260-ac18-951fae3d20f6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:37.244701 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd609a-5017-4260-ac18-951fae3d20f6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:37.246393 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd609a-5017-4260-ac18-951fae3d20f6 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:37.258083 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-f7dd609a-5017-4260-ac18-951fae3d20f6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 14:07:37.258083 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:37.258083 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:37.258083 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:37.258083 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:37.258083 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:37.258083 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:37.258083 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:37.258083 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:37.258083 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:37.258083 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:37.258083 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:37.258083 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 14:07:37.258083 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 14:07:37.258083 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 14:07:37.258083 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 14:07:37.259241 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:37.259241 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:37.259241 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:07:37.259241 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:37.259241 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:07:37.259241 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:37.259241 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:37.259241 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:37.259241 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:37.259241 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:37.259241 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:37.259241 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:37.259241 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:37.259241 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:37.259241 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:37.259241 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:37.259241 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:37.259241 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:37.260492 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:37.260492 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:37.260492 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 14:07:37.260492 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 14:07:37.260492 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 14:07:37.260492 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 14:07:37.260492 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 14:07:37.260492 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 14:07:37.260492 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 14:07:37.260492 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:37.263686 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 348/718] 10.222.0.26 () {56 vars in 1117 bytes} [Fri Aug 5 14:07:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:37.264467 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd609a-5017-4260-ac18-951fae3d20f6 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:37.278762 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4140a0af-3065-462b-8ba8-ec947896cb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:37.279982 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140a0af-3065-462b-8ba8-ec947896cb14 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:37.280710 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140a0af-3065-462b-8ba8-ec947896cb14 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:37.281158 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140a0af-3065-462b-8ba8-ec947896cb14 None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:37.282198 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4140a0af-3065-462b-8ba8-ec947896cb14 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:37.282938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4140a0af-3065-462b-8ba8-ec947896cb14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:37.284495 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4140a0af-3065-462b-8ba8-ec947896cb14 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:37.289158 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:37.289535 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:37.293425 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 371/719] 10.222.0.26 () {56 vars in 1124 bytes} [Fri Aug 5 14:07:37 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:37.294256 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4140a0af-3065-462b-8ba8-ec947896cb14 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:37.390755 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd609a-5017-4260-ac18-951fae3d20f6 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:37.432058 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4140a0af-3065-462b-8ba8-ec947896cb14 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:37.439019 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f38603-a27d-48a7-8051-3ea4e38dbb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:37.439921 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f38603-a27d-48a7-8051-3ea4e38dbb6e None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:37.440341 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f38603-a27d-48a7-8051-3ea4e38dbb6e None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:37.440998 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f38603-a27d-48a7-8051-3ea4e38dbb6e None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:37.442569 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f38603-a27d-48a7-8051-3ea4e38dbb6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:37.443103 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f38603-a27d-48a7-8051-3ea4e38dbb6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:37.444722 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f38603-a27d-48a7-8051-3ea4e38dbb6e None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:37.459496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 349/720] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:37.460884 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f38603-a27d-48a7-8051-3ea4e38dbb6e None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:37.472677 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206d881-be79-4614-b7d5-369edc88fc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:37.474060 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206d881-be79-4614-b7d5-369edc88fc4c None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:37.474479 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206d881-be79-4614-b7d5-369edc88fc4c None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:37.474930 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206d881-be79-4614-b7d5-369edc88fc4c None admin] PATH_INFO: `/v3/services/network` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:37.475748 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6206d881-be79-4614-b7d5-369edc88fc4c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:37.477333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6206d881-be79-4614-b7d5-369edc88fc4c None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:37.487051 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-6206d881-be79-4614-b7d5-369edc88fc4c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 14:07:37.487051 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:37.487051 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 14:07:37.487051 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 14:07:37.487051 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 14:07:37.487051 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 14:07:37.487051 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 14:07:37.487051 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:07:37.487051 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 14:07:37.487051 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:37.487051 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 14:07:37.487051 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:37.487051 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:37.487051 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:37.487051 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:37.487051 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:37.487051 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:37.488845 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:37.488845 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:37.488845 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:37.488845 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:37.488845 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:37.488845 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:37.488845 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 14:07:37.488845 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 14:07:37.488845 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 14:07:37.488845 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 14:07:37.488845 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:37.488845 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:37.488845 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:37.488845 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:37.488845 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:37.488845 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:37.489864 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:37.489864 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:37.489864 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:37.489864 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:37.489864 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:37.489864 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:37.489864 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:37.489864 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:37.489864 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:37.489864 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:37.489864 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 14:07:37.489864 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:07:37.489864 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 14:07:37.489864 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:37.492727 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 372/721] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:07:37 2022] GET /identity/v3/services/network => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:37.493584 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206d881-be79-4614-b7d5-369edc88fc4c None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:37.574007 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:37.575272 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:37.591011 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f38603-a27d-48a7-8051-3ea4e38dbb6e None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:37.630479 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206d881-be79-4614-b7d5-369edc88fc4c None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:37.631022 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf018a-62aa-4fd4-86aa-355b96ba8850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:37.631636 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf018a-62aa-4fd4-86aa-355b96ba8850 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:37.631910 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf018a-62aa-4fd4-86aa-355b96ba8850 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:37.632176 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf018a-62aa-4fd4-86aa-355b96ba8850 None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:37.633164 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf018a-62aa-4fd4-86aa-355b96ba8850 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:37.633466 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf018a-62aa-4fd4-86aa-355b96ba8850 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:37.635195 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf018a-62aa-4fd4-86aa-355b96ba8850 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:37.658697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 350/722] 10.222.0.26 () {56 vars in 1149 bytes} [Fri Aug 5 14:07:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:37.658697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf018a-62aa-4fd4-86aa-355b96ba8850 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:37.684763 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c3ae99-0825-4916-81df-2bd11d371099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:37.686287 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3ae99-0825-4916-81df-2bd11d371099 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:37.686746 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3ae99-0825-4916-81df-2bd11d371099 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:37.687450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3ae99-0825-4916-81df-2bd11d371099 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:37.696784 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c3ae99-0825-4916-81df-2bd11d371099 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:37.698591 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c3ae99-0825-4916-81df-2bd11d371099 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:37.715520 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-57c3ae99-0825-4916-81df-2bd11d371099 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:37.715520 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:37.715520 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:37.715520 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:37.715520 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:37.715520 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:37.715520 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:37.715520 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:37.715520 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:37.715520 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:37.715520 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:37.715520 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:37.715520 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 14:07:37.715520 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 14:07:37.715520 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 14:07:37.716477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 14:07:37.716477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:37.716477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:37.716477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:37.716477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:37.716477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:37.716477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:37.716477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:37.716477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:37.716477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:37.716477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:37.716477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:37.716477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:37.716477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:37.716477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:37.716477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:37.716477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:37.717371 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 14:07:37.717371 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 14:07:37.717371 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:07:37.717371 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:07:37.717371 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:07:37.717371 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:07:37.717371 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:37.717371 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:37.718375 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 373/723] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:07:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:37.718877 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c3ae99-0825-4916-81df-2bd11d371099 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:37.777770 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf018a-62aa-4fd4-86aa-355b96ba8850 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:37.815459 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4810e6-57ab-4092-bd52-20e9da37a461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:37.816943 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4810e6-57ab-4092-bd52-20e9da37a461 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:37.817231 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4810e6-57ab-4092-bd52-20e9da37a461 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:37.817504 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4810e6-57ab-4092-bd52-20e9da37a461 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:37.818190 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4810e6-57ab-4092-bd52-20e9da37a461 None admin] RBAC: Adding query filter params (name=network) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:37.818492 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4810e6-57ab-4092-bd52-20e9da37a461 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:37.820699 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4810e6-57ab-4092-bd52-20e9da37a461 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:37.826536 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 351/724] 10.222.0.26 () {56 vars in 1112 bytes} [Fri Aug 5 14:07:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:37.827885 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4810e6-57ab-4092-bd52-20e9da37a461 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:37.840209 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:37.847798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c3ae99-0825-4916-81df-2bd11d371099 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:37.872506 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:37.885833 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898700b-fdd1-4888-87e9-0e50fd781f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:37.886806 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898700b-fdd1-4888-87e9-0e50fd781f53 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:37.887323 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898700b-fdd1-4888-87e9-0e50fd781f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:37.887965 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898700b-fdd1-4888-87e9-0e50fd781f53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:37.951591 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4810e6-57ab-4092-bd52-20e9da37a461 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:37.964192 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a898700b-fdd1-4888-87e9-0e50fd781f53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:37.966154 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a898700b-fdd1-4888-87e9-0e50fd781f53 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:37.986396 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe80e831-94b6-4683-b028-cf343fa2f2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:37.986851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe80e831-94b6-4683-b028-cf343fa2f2bf None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:37.987130 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe80e831-94b6-4683-b028-cf343fa2f2bf None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:37.987405 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe80e831-94b6-4683-b028-cf343fa2f2bf None admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:37.988901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe80e831-94b6-4683-b028-cf343fa2f2bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:37.989212 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe80e831-94b6-4683-b028-cf343fa2f2bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:37.990647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe80e831-94b6-4683-b028-cf343fa2f2bf None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:38.003711 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 352/725] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Aug 5 14:07:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:38.004651 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe80e831-94b6-4683-b028-cf343fa2f2bf None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:38.072467 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:38.084692 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 374/726] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:07:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 14:07:38.085869 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898700b-fdd1-4888-87e9-0e50fd781f53 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:38.109862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe80e831-94b6-4683-b028-cf343fa2f2bf None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:38.113882 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:38.146571 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba59b9da-e537-4209-8439-bc9eb5b8f9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:38.147180 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba59b9da-e537-4209-8439-bc9eb5b8f9ed None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:38.147459 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba59b9da-e537-4209-8439-bc9eb5b8f9ed None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:38.147730 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba59b9da-e537-4209-8439-bc9eb5b8f9ed None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:38.148427 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba59b9da-e537-4209-8439-bc9eb5b8f9ed None admin] RBAC: Adding query filter params (type=network) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:38.148744 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba59b9da-e537-4209-8439-bc9eb5b8f9ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:38.150102 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba59b9da-e537-4209-8439-bc9eb5b8f9ed None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:38.156233 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 353/727] 10.222.0.26 () {56 vars in 1112 bytes} [Fri Aug 5 14:07:38 2022] GET /identity/v3/services?type=network => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:38.157185 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba59b9da-e537-4209-8439-bc9eb5b8f9ed None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:38.198892 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898700b-fdd1-4888-87e9-0e50fd781f53 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:38.237517 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfd1bed-62a8-4d51-8619-bbb35bc4e345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:38.239214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd1bed-62a8-4d51-8619-bbb35bc4e345 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:38.239692 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd1bed-62a8-4d51-8619-bbb35bc4e345 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:38.240176 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd1bed-62a8-4d51-8619-bbb35bc4e345 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:38.241214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfd1bed-62a8-4d51-8619-bbb35bc4e345 None admin] RBAC: Adding query filter params (role.id=299c794c7acb4a5ca36288c3191aad10, scope.project.id=319973e011c54b4ca1f2198c8a5e135b, user.id=5ec4940c99fc4fa8a5bc7cdabc4558fb) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:38.241722 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfd1bed-62a8-4d51-8619-bbb35bc4e345 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:38.243467 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfd1bed-62a8-4d51-8619-bbb35bc4e345 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:38.250353 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 375/728] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:07:38 2022] GET /identity/v3/role_assignments?user.id=5ec4940c99fc4fa8a5bc7cdabc4558fb&scope.project.id=319973e011c54b4ca1f2198c8a5e135b&role.id=299c794c7acb4a5ca36288c3191aad10 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:38.324713 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba59b9da-e537-4209-8439-bc9eb5b8f9ed None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:38.356090 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64825e5a-c5b0-446a-854a-5cffa6521b99 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:38.356435 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64825e5a-c5b0-446a-854a-5cffa6521b99 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:38.356676 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64825e5a-c5b0-446a-854a-5cffa6521b99 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:38.358375 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 376/729] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:07:38.364722 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41290e-5709-4313-8653-92cce0226b87 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:38.365025 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41290e-5709-4313-8653-92cce0226b87 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:38.365271 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41290e-5709-4313-8653-92cce0226b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:38.386263 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:38.387410 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-9d41290e-5709-4313-8653-92cce0226b87 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:38.388852 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-9d41290e-5709-4313-8653-92cce0226b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:38.399500 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709d650-7d91-484e-b03d-aa6c8c367858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:38.400165 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709d650-7d91-484e-b03d-aa6c8c367858 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:38.400469 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709d650-7d91-484e-b03d-aa6c8c367858 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:38.400745 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709d650-7d91-484e-b03d-aa6c8c367858 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:38.419897 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7709d650-7d91-484e-b03d-aa6c8c367858 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:38.421409 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7709d650-7d91-484e-b03d-aa6c8c367858 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:38.438754 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-7709d650-7d91-484e-b03d-aa6c8c367858 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:38.438754 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:38.438754 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:38.438754 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:38.438754 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:38.438754 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:38.438754 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:38.438754 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:38.438754 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:38.438754 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:38.438754 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:38.438754 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:38.438754 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:38.438754 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:38.438754 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:38.439882 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:38.439882 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:38.439882 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:38.439882 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:38.439882 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:38.439882 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:38.439882 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:38.439882 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:38.439882 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:38.439882 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:38.439882 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:38.439882 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:38.439882 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:38.439882 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:38.439882 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:38.439882 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:38.439882 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:38.441068 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:38.441068 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:38.441068 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:38.441068 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:38.441068 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 354/730] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:38.449615 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709d650-7d91-484e-b03d-aa6c8c367858 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:38.497004 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 377/731] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 14:07:38.521051 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41290e-5709-4313-8653-92cce0226b87 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:38.553909 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:38.576548 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709d650-7d91-484e-b03d-aa6c8c367858 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:38.612394 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f256e587-97be-43e6-9260-5bdf5fd8e770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:38.614450 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256e587-97be-43e6-9260-5bdf5fd8e770 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:38.614450 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256e587-97be-43e6-9260-5bdf5fd8e770 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:38.614450 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256e587-97be-43e6-9260-5bdf5fd8e770 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:38.615148 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f256e587-97be-43e6-9260-5bdf5fd8e770 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:38.615441 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f256e587-97be-43e6-9260-5bdf5fd8e770 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:38.616862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f256e587-97be-43e6-9260-5bdf5fd8e770 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:38.630818 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 355/732] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:38.636214 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f256e587-97be-43e6-9260-5bdf5fd8e770 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:38.652117 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41290e-5709-4313-8653-92cce0226b87 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:38.727950 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302b413-b31c-41c9-b4e3-46906ed6f48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:38.729034 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302b413-b31c-41c9-b4e3-46906ed6f48a None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:38.729514 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302b413-b31c-41c9-b4e3-46906ed6f48a None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:38.729979 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302b413-b31c-41c9-b4e3-46906ed6f48a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:38.740056 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:38.752817 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4302b413-b31c-41c9-b4e3-46906ed6f48a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:07:38.758272 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-4302b413-b31c-41c9-b4e3-46906ed6f48a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:07:38.758272 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:38.758272 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:38.758272 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:38.758272 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:38.758272 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:38.758272 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:38.758272 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:38.758272 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:38.758272 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:38.758272 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:38.758272 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:38.758272 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:07:38.758272 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:07:38.758272 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:07:38.758272 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:07:38.759200 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:07:38.759200 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:07:38.759200 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:07:38.759200 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:07:38.759200 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:38.759200 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:38.759200 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:38.759200 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:38.759200 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:38.759200 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:38.759200 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:38.759200 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:38.759200 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:38.759200 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:38.759200 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:38.759200 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:38.760052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:38.760052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:38.760052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:38.760052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:38.760052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:07:38.760052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:07:38.760052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:07:38.760052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:07:38.760052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:07:38.760052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:07:38.760052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:07:38.760052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:38.772031 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 378/733] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Aug 5 14:07:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:38.797333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302b413-b31c-41c9-b4e3-46906ed6f48a None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:38.852269 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f256e587-97be-43e6-9260-5bdf5fd8e770 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:38.915859 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde56d95-0fbf-4e89-8f44-64ba4d5cb055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:38.916603 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde56d95-0fbf-4e89-8f44-64ba4d5cb055 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:38.916883 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde56d95-0fbf-4e89-8f44-64ba4d5cb055 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:38.917157 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde56d95-0fbf-4e89-8f44-64ba4d5cb055 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:38.924517 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde56d95-0fbf-4e89-8f44-64ba4d5cb055 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:38.926215 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde56d95-0fbf-4e89-8f44-64ba4d5cb055 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:38.938799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-cde56d95-0fbf-4e89-8f44-64ba4d5cb055 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:38.938799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:38.938799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:38.938799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:38.938799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:38.938799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:38.938799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:38.938799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:38.938799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:38.938799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:38.938799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:38.938799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:38.938799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 14:07:38.938799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 14:07:38.938799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 14:07:38.939717 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 14:07:38.939717 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:38.939717 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:38.939717 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:38.939717 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:38.939717 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:38.939717 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:38.939717 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:38.939717 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:38.939717 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:38.939717 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:38.939717 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:38.939717 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:38.939717 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:38.939717 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:38.939717 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:38.939717 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:38.940630 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 14:07:38.940630 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 14:07:38.940630 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:07:38.940630 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:07:38.940630 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:07:38.940630 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:07:38.940630 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:38.940630 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:38.940630 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 356/734] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:07:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:07:38.946061 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde56d95-0fbf-4e89-8f44-64ba4d5cb055 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:38.975157 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:39.015514 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302b413-b31c-41c9-b4e3-46906ed6f48a None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:39.084113 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2077eff-fb60-473b-897f-b3559778a5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:39.084759 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2077eff-fb60-473b-897f-b3559778a5cb None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:39.085036 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2077eff-fb60-473b-897f-b3559778a5cb None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:39.085307 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2077eff-fb60-473b-897f-b3559778a5cb None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:39.086023 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2077eff-fb60-473b-897f-b3559778a5cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:39.086323 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2077eff-fb60-473b-897f-b3559778a5cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:39.087685 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2077eff-fb60-473b-897f-b3559778a5cb None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:39.106234 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 379/735] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:07:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:39.112333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2077eff-fb60-473b-897f-b3559778a5cb None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:39.146334 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde56d95-0fbf-4e89-8f44-64ba4d5cb055 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:39.149329 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:39.201871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcdbefe-861f-4b43-af19-c9435f5230a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:39.202560 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdbefe-861f-4b43-af19-c9435f5230a7 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:39.202838 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdbefe-861f-4b43-af19-c9435f5230a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:39.203106 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdbefe-861f-4b43-af19-c9435f5230a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:39.203858 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcdbefe-861f-4b43-af19-c9435f5230a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:39.204161 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcdbefe-861f-4b43-af19-c9435f5230a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:39.205610 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcdbefe-861f-4b43-af19-c9435f5230a7 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:39.236239 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 357/736] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Aug 5 14:07:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:39.243431 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcdbefe-861f-4b43-af19-c9435f5230a7 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:39.285389 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2077eff-fb60-473b-897f-b3559778a5cb None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:39.354311 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4df26-9525-4dc7-8d28-f4c959fba5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:39.355377 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4df26-9525-4dc7-8d28-f4c959fba5b9 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:39.355856 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4df26-9525-4dc7-8d28-f4c959fba5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:39.356325 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4df26-9525-4dc7-8d28-f4c959fba5b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:39.365836 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:39.368847 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c4df26-9525-4dc7-8d28-f4c959fba5b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:39.370478 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c4df26-9525-4dc7-8d28-f4c959fba5b9 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:39.386155 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-b5c4df26-9525-4dc7-8d28-f4c959fba5b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:39.386155 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:39.386155 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:39.386155 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:39.386155 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:39.386155 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:39.386155 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:39.386155 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:39.386155 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:39.386155 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:39.386155 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:39.386155 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:39.386155 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:39.386155 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:39.386155 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:39.387074 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:39.387074 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:39.387074 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:39.387074 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:39.387074 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:39.387074 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:39.387074 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:39.387074 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:39.387074 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:39.387074 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:39.387074 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:39.387074 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:39.387074 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:39.387074 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:39.387074 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:39.387074 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:39.387074 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:39.387954 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:39.387954 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:39.387954 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:39.387954 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:39.390099 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 380/737] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:39.405597 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4df26-9525-4dc7-8d28-f4c959fba5b9 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:39.443521 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcdbefe-861f-4b43-af19-c9435f5230a7 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:39.485420 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4559a65c-3485-4b35-9a73-e60e1c51e458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:39.486115 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559a65c-3485-4b35-9a73-e60e1c51e458 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:39.486321 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559a65c-3485-4b35-9a73-e60e1c51e458 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:39.486576 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559a65c-3485-4b35-9a73-e60e1c51e458 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:39.487360 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4559a65c-3485-4b35-9a73-e60e1c51e458 None admin] RBAC: Adding query filter params (role.id=ec18a2850fd540a39d4a935348bb4474, scope.project.id=319973e011c54b4ca1f2198c8a5e135b, user.id=17e3b509a2de4159bf234594573f5e18) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:39.487740 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4559a65c-3485-4b35-9a73-e60e1c51e458 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:39.489285 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4559a65c-3485-4b35-9a73-e60e1c51e458 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:39.497502 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 358/738] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:07:39 2022] GET /identity/v3/role_assignments?user.id=17e3b509a2de4159bf234594573f5e18&scope.project.id=319973e011c54b4ca1f2198c8a5e135b&role.id=ec18a2850fd540a39d4a935348bb4474 => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:39.547201 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4df26-9525-4dc7-8d28-f4c959fba5b9 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:39.597091 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a51fc-dd78-4737-a38a-e8bfe13fedbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:39.597991 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a51fc-dd78-4737-a38a-e8bfe13fedbb None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:39.598444 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a51fc-dd78-4737-a38a-e8bfe13fedbb None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:39.598875 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a51fc-dd78-4737-a38a-e8bfe13fedbb None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:39.599753 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4a51fc-dd78-4737-a38a-e8bfe13fedbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:39.600220 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4a51fc-dd78-4737-a38a-e8bfe13fedbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:39.601873 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4a51fc-dd78-4737-a38a-e8bfe13fedbb None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:39.615933 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 381/739] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:39.622150 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4559a65c-3485-4b35-9a73-e60e1c51e458 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:39.675940 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:39.729926 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f27bfc-cbf0-469e-82ee-a66b9a9dbc05 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:39.730581 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f27bfc-cbf0-469e-82ee-a66b9a9dbc05 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:39.731081 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f27bfc-cbf0-469e-82ee-a66b9a9dbc05 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:39.732946 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 382/740] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:07:39.739035 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d2fe7-b854-4b46-a428-17440cb80bd1 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:39.739274 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d2fe7-b854-4b46-a428-17440cb80bd1 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:39.739575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d2fe7-b854-4b46-a428-17440cb80bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:39.765473 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-1e2d2fe7-b854-4b46-a428-17440cb80bd1 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:39.767021 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-1e2d2fe7-b854-4b46-a428-17440cb80bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:39.768665 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4559a65c-3485-4b35-9a73-e60e1c51e458 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:39.805670 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9ea89-3acc-4fef-a775-d815b8b531b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:39.806276 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9ea89-3acc-4fef-a775-d815b8b531b3 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:39.806551 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9ea89-3acc-4fef-a775-d815b8b531b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:39.806820 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9ea89-3acc-4fef-a775-d815b8b531b3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:39.814200 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde9ea89-3acc-4fef-a775-d815b8b531b3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:39.816626 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde9ea89-3acc-4fef-a775-d815b8b531b3 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:39.826430 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:39.837921 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-bde9ea89-3acc-4fef-a775-d815b8b531b3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 14:07:39.837921 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:39.837921 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:39.837921 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:39.837921 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:39.837921 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:39.837921 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:39.837921 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:39.837921 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:39.837921 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:39.837921 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:39.837921 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:39.837921 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 14:07:39.837921 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 14:07:39.837921 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 14:07:39.837921 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 14:07:39.846451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:39.846451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:39.846451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:07:39.846451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:39.846451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:07:39.846451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:39.846451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:39.846451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:39.846451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:39.846451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:39.846451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:39.846451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:39.846451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:39.846451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:39.846451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:39.846451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:39.846451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:39.846451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:39.847459 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:39.847459 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:39.847459 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 14:07:39.847459 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 14:07:39.847459 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 14:07:39.847459 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 14:07:39.847459 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 14:07:39.847459 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 14:07:39.847459 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 14:07:39.847459 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:39.847459 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 359/741] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Aug 5 14:07:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:39.847459 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9ea89-3acc-4fef-a775-d815b8b531b3 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:39.871328 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 383/742] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:07:39.910078 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d2fe7-b854-4b46-a428-17440cb80bd1 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:39.958640 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:39.966726 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9ea89-3acc-4fef-a775-d815b8b531b3 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:40.005521 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c042e-c450-45f5-839d-28261e922aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:40.006237 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c042e-c450-45f5-839d-28261e922aec None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:40.006792 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c042e-c450-45f5-839d-28261e922aec None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:40.007136 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c042e-c450-45f5-839d-28261e922aec None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:40.008065 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c042e-c450-45f5-839d-28261e922aec None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:40.008461 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c042e-c450-45f5-839d-28261e922aec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:40.009957 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c042e-c450-45f5-839d-28261e922aec None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:40.029876 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 360/743] 10.222.0.26 () {56 vars in 1135 bytes} [Fri Aug 5 14:07:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:40.035468 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c042e-c450-45f5-839d-28261e922aec None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:40.129193 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d2fe7-b854-4b46-a428-17440cb80bd1 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:40.163993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172a28f-5af7-4998-9448-baf17f400629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:40.164636 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172a28f-5af7-4998-9448-baf17f400629 None admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:40.164911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172a28f-5af7-4998-9448-baf17f400629 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:40.165177 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172a28f-5af7-4998-9448-baf17f400629 None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:40.165884 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7172a28f-5af7-4998-9448-baf17f400629 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:40.167226 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7172a28f-5af7-4998-9448-baf17f400629 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:40.211082 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c042e-c450-45f5-839d-28261e922aec None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:40.248289 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632fee52-9bc5-4185-81c5-65f6ceddb7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:40.248933 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632fee52-9bc5-4185-81c5-65f6ceddb7a6 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:40.249271 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632fee52-9bc5-4185-81c5-65f6ceddb7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:40.249540 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632fee52-9bc5-4185-81c5-65f6ceddb7a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:40.265621 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:40.285502 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 384/744] 10.222.0.26 () {60 vars in 1141 bytes} [Fri Aug 5 14:07:39 2022] POST /identity/v3/services => generated 251 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:07:40.330494 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632fee52-9bc5-4185-81c5-65f6ceddb7a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:40.332048 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632fee52-9bc5-4185-81c5-65f6ceddb7a6 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:40.440957 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 361/745] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:07:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 14:07:40.446446 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172a28f-5af7-4998-9448-baf17f400629 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:40.500764 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ca61c-7f18-44eb-b2fb-4fc7cd6f16dd None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:40.501097 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ca61c-7f18-44eb-b2fb-4fc7cd6f16dd None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:40.501427 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ca61c-7f18-44eb-b2fb-4fc7cd6f16dd None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:40.502791 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 362/746] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:40.510947 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38a94d-91c6-4c33-9025-6f45719837d9 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:40.510947 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38a94d-91c6-4c33-9025-6f45719837d9 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:40.510947 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38a94d-91c6-4c33-9025-6f45719837d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:40.524807 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:40.557402 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-1c38a94d-91c6-4c33-9025-6f45719837d9 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:40.558609 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-1c38a94d-91c6-4c33-9025-6f45719837d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:40.602769 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172a28f-5af7-4998-9448-baf17f400629 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:40.613891 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:40.642772 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56801744-fc09-4279-a553-0ae046432324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:40.644313 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56801744-fc09-4279-a553-0ae046432324 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:40.645717 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56801744-fc09-4279-a553-0ae046432324 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:40.645997 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56801744-fc09-4279-a553-0ae046432324 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:40.652678 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 363/747] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:40 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 14:07:40.663564 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56801744-fc09-4279-a553-0ae046432324 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:40.665146 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56801744-fc09-4279-a553-0ae046432324 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:40.678083 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38a94d-91c6-4c33-9025-6f45719837d9 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:40.684443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-56801744-fc09-4279-a553-0ae046432324 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:40.684443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:40.684443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:40.684443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:40.684443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:40.684443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:40.684443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:40.684443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:40.684443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:40.684443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:40.684443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:40.684443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:40.684443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:40.684443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:40.684443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:40.685388 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:40.685388 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:40.685388 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:40.685388 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:40.685388 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:40.685388 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:40.685388 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:40.685388 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:40.685388 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:40.685388 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:40.685388 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:40.685388 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:40.685388 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:40.685388 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:40.685388 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:40.685388 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:40.685388 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:40.686758 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:40.686758 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:40.686758 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:40.686758 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:40.686758 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 385/748] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:40.691657 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56801744-fc09-4279-a553-0ae046432324 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:40.812334 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38a94d-91c6-4c33-9025-6f45719837d9 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:40.814068 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56801744-fc09-4279-a553-0ae046432324 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:40.829038 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:40.856760 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f11e9a-d260-444e-866e-0f81b1b16917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:40.857721 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f11e9a-d260-444e-866e-0f81b1b16917 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:40.865044 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f11e9a-d260-444e-866e-0f81b1b16917 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:40.865158 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1187920-8399-44d6-a848-62883fc0ec26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:40.865268 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f11e9a-d260-444e-866e-0f81b1b16917 None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:40.865344 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1187920-8399-44d6-a848-62883fc0ec26 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:40.865344 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1187920-8399-44d6-a848-62883fc0ec26 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:40.865485 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f11e9a-d260-444e-866e-0f81b1b16917 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:40.865553 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1187920-8399-44d6-a848-62883fc0ec26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:40.865628 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f11e9a-d260-444e-866e-0f81b1b16917 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:40.865628 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f11e9a-d260-444e-866e-0f81b1b16917 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:40.872547 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1187920-8399-44d6-a848-62883fc0ec26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:07:40.876620 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 386/749] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:40.878732 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1cab8-4065-4c50-9479-0d8493fa062c None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:40.879544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-f1187920-8399-44d6-a848-62883fc0ec26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:07:40.879544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:40.879544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:40.879544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:40.879544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:40.879544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:40.879544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:40.879544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:40.879544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:40.879544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:40.879544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:40.879544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:40.879544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:07:40.879544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:07:40.879544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:07:40.879544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:07:40.880681 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:07:40.880681 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:07:40.880681 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:07:40.880681 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:07:40.880681 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:40.880681 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:40.880681 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:40.880681 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:40.880681 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:40.880681 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:40.880681 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:40.880681 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:40.880681 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:40.880681 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:40.880681 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:40.880681 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:40.881795 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:40.881795 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:40.881795 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:40.881795 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:40.881795 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:07:40.881795 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:07:40.881795 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:07:40.881795 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:07:40.881795 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:07:40.881795 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:07:40.881795 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:07:40.881795 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:40.882835 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1cab8-4065-4c50-9479-0d8493fa062c None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:40.883252 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1cab8-4065-4c50-9479-0d8493fa062c None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:40.888244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 364/750] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Aug 5 14:07:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:07:40.888244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1187920-8399-44d6-a848-62883fc0ec26 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:40.891961 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 387/751] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:40 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:40.893310 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a1cab8-4065-4c50-9479-0d8493fa062c None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:40.926233 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:41.017946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1187920-8399-44d6-a848-62883fc0ec26 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:41.039438 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a1cab8-4065-4c50-9479-0d8493fa062c None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:41.069308 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240dfa6-f8ed-4e5b-8790-f724b480d593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:41.070516 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240dfa6-f8ed-4e5b-8790-f724b480d593 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:41.071315 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240dfa6-f8ed-4e5b-8790-f724b480d593 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:41.072180 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240dfa6-f8ed-4e5b-8790-f724b480d593 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:41.076872 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b043f388-252d-4dbd-9100-e6369441379d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:41.077577 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043f388-252d-4dbd-9100-e6369441379d None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:41.077922 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043f388-252d-4dbd-9100-e6369441379d None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:41.078264 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043f388-252d-4dbd-9100-e6369441379d None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:41.079043 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b043f388-252d-4dbd-9100-e6369441379d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:41.079408 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b043f388-252d-4dbd-9100-e6369441379d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:41.080873 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b043f388-252d-4dbd-9100-e6369441379d None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:41.082424 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240dfa6-f8ed-4e5b-8790-f724b480d593 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:41.084886 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240dfa6-f8ed-4e5b-8790-f724b480d593 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:41.094073 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 388/752] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:07:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:41.095628 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd05b2-5184-4125-a441-31031118d772 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:41.095763 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd05b2-5184-4125-a441-31031118d772 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:41.096005 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd05b2-5184-4125-a441-31031118d772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:41.099262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-c240dfa6-f8ed-4e5b-8790-f724b480d593 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:41.099262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:41.099262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:41.099262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:41.099262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:41.099262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:41.099262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:41.099262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:41.099262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:41.099262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:41.099262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:41.099262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:41.099262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 14:07:41.099262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 14:07:41.099262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 14:07:41.100128 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 14:07:41.100128 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:41.100128 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:41.100128 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:41.100128 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:41.100128 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:41.100128 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:41.100128 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:41.100128 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:41.100128 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:41.100128 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:41.100128 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:41.100128 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:41.100128 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:41.100128 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:41.100128 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:41.100128 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:41.101459 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 14:07:41.101459 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 14:07:41.101459 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:07:41.101459 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:07:41.101459 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:07:41.101459 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:07:41.101459 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:41.101459 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:41.101459 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 365/753] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:07:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:41.101941 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240dfa6-f8ed-4e5b-8790-f724b480d593 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:41.121069 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-78cd05b2-5184-4125-a441-31031118d772 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:41.122272 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-78cd05b2-5184-4125-a441-31031118d772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:41.131200 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:41.200802 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 389/754] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:41 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 14:07:41.201734 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cd05b2-5184-4125-a441-31031118d772 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:41.220826 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:41.230418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240dfa6-f8ed-4e5b-8790-f724b480d593 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:41.277807 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d3895-87ac-4c32-b8f0-d466af2ae8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:41.279052 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d3895-87ac-4c32-b8f0-d466af2ae8cb None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:41.279734 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d3895-87ac-4c32-b8f0-d466af2ae8cb None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:41.280161 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d3895-87ac-4c32-b8f0-d466af2ae8cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:41.294568 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d3895-87ac-4c32-b8f0-d466af2ae8cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:41.296265 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d3895-87ac-4c32-b8f0-d466af2ae8cb None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:41.316529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-362d3895-87ac-4c32-b8f0-d466af2ae8cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:41.316529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:41.316529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:41.316529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:41.316529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:41.316529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:41.316529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:41.316529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:41.316529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:41.316529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:41.316529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:41.316529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:41.316529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:41.316529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:41.316529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:41.317731 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:41.317731 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:41.317731 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:41.317731 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:41.317731 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:41.317731 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:41.317731 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:41.317731 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:41.317731 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:41.317731 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:41.317731 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:41.317731 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:41.317731 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:41.317731 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:41.317731 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:41.317731 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:41.317731 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:41.318832 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:41.318832 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:41.318832 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:41.318832 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:41.321062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 366/755] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:41.322378 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d3895-87ac-4c32-b8f0-d466af2ae8cb None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:41.328243 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cd05b2-5184-4125-a441-31031118d772 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:41.368069 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed5ee4-765d-40c4-b9dd-f6043d244fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:41.368971 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed5ee4-765d-40c4-b9dd-f6043d244fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:41.369398 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed5ee4-765d-40c4-b9dd-f6043d244fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:41.369764 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed5ee4-765d-40c4-b9dd-f6043d244fb2 None admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:41.371078 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed5ee4-765d-40c4-b9dd-f6043d244fb2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:41.371377 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed5ee4-765d-40c4-b9dd-f6043d244fb2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:41.373000 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed5ee4-765d-40c4-b9dd-f6043d244fb2 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:41.391318 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 390/756] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Aug 5 14:07:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:41.391986 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed5ee4-765d-40c4-b9dd-f6043d244fb2 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:41.395624 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:41.451841 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d3895-87ac-4c32-b8f0-d466af2ae8cb None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:41.498831 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8393a4ce-c603-4423-a466-33d2844bfb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:41.500013 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393a4ce-c603-4423-a466-33d2844bfb3f None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:41.500730 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393a4ce-c603-4423-a466-33d2844bfb3f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:41.501182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393a4ce-c603-4423-a466-33d2844bfb3f None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:41.502181 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8393a4ce-c603-4423-a466-33d2844bfb3f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:41.504387 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8393a4ce-c603-4423-a466-33d2844bfb3f None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:41.505832 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed5ee4-765d-40c4-b9dd-f6043d244fb2 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:41.545861 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83ecbc-8581-4860-8288-2b9067b35fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:41.546585 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83ecbc-8581-4860-8288-2b9067b35fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:41.546874 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83ecbc-8581-4860-8288-2b9067b35fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:41.547143 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83ecbc-8581-4860-8288-2b9067b35fd8 None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:41.547891 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f83ecbc-8581-4860-8288-2b9067b35fd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:41.548195 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f83ecbc-8581-4860-8288-2b9067b35fd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:41.549658 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f83ecbc-8581-4860-8288-2b9067b35fd8 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:41.562850 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 391/757] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:41.563540 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83ecbc-8581-4860-8288-2b9067b35fd8 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:41.590630 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 367/758] 10.222.0.26 () {60 vars in 1141 bytes} [Fri Aug 5 14:07:41 2022] POST /identity/v3/services => generated 240 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:07:41.590630 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8393a4ce-c603-4423-a466-33d2844bfb3f None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:41.611176 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:41.682321 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:41.752203 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83ecbc-8581-4860-8288-2b9067b35fd8 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:41.793668 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8393a4ce-c603-4423-a466-33d2844bfb3f None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:41.813188 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eccb02-20d4-4d5f-b89c-9b6fd7997f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:41.814161 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eccb02-20d4-4d5f-b89c-9b6fd7997f81 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:41.814641 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eccb02-20d4-4d5f-b89c-9b6fd7997f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:41.815112 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eccb02-20d4-4d5f-b89c-9b6fd7997f81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:41.816190 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eccb02-20d4-4d5f-b89c-9b6fd7997f81 None admin] RBAC: Adding query filter params (role.id=299c794c7acb4a5ca36288c3191aad10, scope.project.id=319973e011c54b4ca1f2198c8a5e135b, user.id=dddd7eea67f34f0cbb5f6a1cd85f1fc5) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:41.817080 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eccb02-20d4-4d5f-b89c-9b6fd7997f81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:41.818742 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eccb02-20d4-4d5f-b89c-9b6fd7997f81 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:41.825178 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 392/759] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:07:41 2022] GET /identity/v3/role_assignments?user.id=dddd7eea67f34f0cbb5f6a1cd85f1fc5&scope.project.id=319973e011c54b4ca1f2198c8a5e135b&role.id=299c794c7acb4a5ca36288c3191aad10 => generated 259 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:41.836200 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d2071-75d2-4077-aa32-61b2e83c6f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:41.836902 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d2071-75d2-4077-aa32-61b2e83c6f4f None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:41.837283 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d2071-75d2-4077-aa32-61b2e83c6f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:41.837552 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d2071-75d2-4077-aa32-61b2e83c6f4f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:41.846471 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78d2071-75d2-4077-aa32-61b2e83c6f4f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:41.848981 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78d2071-75d2-4077-aa32-61b2e83c6f4f None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:41.863457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-b78d2071-75d2-4077-aa32-61b2e83c6f4f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 14:07:41.863457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:41.863457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:41.863457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:41.863457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:41.863457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:41.863457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:41.863457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:41.863457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:41.863457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:41.863457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:41.863457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:41.863457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 14:07:41.863457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 14:07:41.863457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 14:07:41.863457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 14:07:41.864534 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:41.864534 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:41.864534 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:07:41.864534 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:41.864534 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:07:41.864534 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:41.864534 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:41.864534 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:41.864534 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:41.864534 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:41.864534 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:41.864534 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:41.864534 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:41.864534 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:41.864534 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:41.864534 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:41.864534 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:41.864534 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:41.865477 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:41.865477 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:41.865477 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 14:07:41.865477 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 14:07:41.865477 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 14:07:41.865477 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 14:07:41.865477 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 14:07:41.865477 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 14:07:41.865477 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 14:07:41.865477 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:41.865477 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 368/760] 10.222.0.26 () {56 vars in 1093 bytes} [Fri Aug 5 14:07:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:41.871147 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eccb02-20d4-4d5f-b89c-9b6fd7997f81 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:41.901425 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8757c05-b3c2-40a0-9f8e-4f047d2b3904 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:41.901849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8757c05-b3c2-40a0-9f8e-4f047d2b3904 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:41.902004 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8757c05-b3c2-40a0-9f8e-4f047d2b3904 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:41.903611 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 369/761] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:41.909940 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce85385-41c6-4ab3-840c-3fba7df836c3 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:41.910399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce85385-41c6-4ab3-840c-3fba7df836c3 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:41.910708 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce85385-41c6-4ab3-840c-3fba7df836c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:41.937437 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-4ce85385-41c6-4ab3-840c-3fba7df836c3 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:41.940230 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-4ce85385-41c6-4ab3-840c-3fba7df836c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:42.023346 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:42.026748 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eccb02-20d4-4d5f-b89c-9b6fd7997f81 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:42.034997 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 370/762] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:07:42.055482 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:42.064970 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c659a1b-8e0c-4410-ba8d-74c329d1e307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:42.065995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c659a1b-8e0c-4410-ba8d-74c329d1e307 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:42.066429 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c659a1b-8e0c-4410-ba8d-74c329d1e307 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:42.066918 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c659a1b-8e0c-4410-ba8d-74c329d1e307 None admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:42.067868 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c659a1b-8e0c-4410-ba8d-74c329d1e307 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:42.068328 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c659a1b-8e0c-4410-ba8d-74c329d1e307 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:42.070191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c659a1b-8e0c-4410-ba8d-74c329d1e307 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:42.075420 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce85385-41c6-4ab3-840c-3fba7df836c3 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:42.092091 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 393/763] 10.222.0.26 () {56 vars in 1133 bytes} [Fri Aug 5 14:07:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:42.097485 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c659a1b-8e0c-4410-ba8d-74c329d1e307 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:42.275617 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce85385-41c6-4ab3-840c-3fba7df836c3 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:42.290175 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c659a1b-8e0c-4410-ba8d-74c329d1e307 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:42.334426 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641da42-dfae-45db-8a20-a44adc0ea916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:42.335313 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641da42-dfae-45db-8a20-a44adc0ea916 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:42.335742 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641da42-dfae-45db-8a20-a44adc0ea916 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:42.336162 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641da42-dfae-45db-8a20-a44adc0ea916 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:42.354965 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b641da42-dfae-45db-8a20-a44adc0ea916 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:42.356990 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b641da42-dfae-45db-8a20-a44adc0ea916 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:42.358660 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e52a98-3db0-4058-aeed-aec5c1a0063f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:42.359476 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e52a98-3db0-4058-aeed-aec5c1a0063f None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:42.359931 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e52a98-3db0-4058-aeed-aec5c1a0063f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:42.360430 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e52a98-3db0-4058-aeed-aec5c1a0063f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:42.380985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e52a98-3db0-4058-aeed-aec5c1a0063f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:07:42.391450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-b641da42-dfae-45db-8a20-a44adc0ea916 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:42.391450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:42.391450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:42.391450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:42.391450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:42.391450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:42.391450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:42.391450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:42.391450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:42.391450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:42.391450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:42.391450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:42.391450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:42.391450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:42.391450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:42.392585 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:42.392585 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:42.392585 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:42.392585 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:42.392585 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:42.392585 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:42.392585 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:42.392585 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:42.392585 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:42.392585 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:42.392585 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:42.392585 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:42.392585 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:42.392585 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:42.392585 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:42.392585 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:42.392585 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:42.393741 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:42.393741 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:42.393741 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:42.393741 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:42.396125 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 394/764] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:07:42.396451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-43e52a98-3db0-4058-aeed-aec5c1a0063f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:07:42.396451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:42.396451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:42.396451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:42.396451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:42.396451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:42.396451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:42.396451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:42.396451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:42.396451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:42.396451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:42.396451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:42.396451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:07:42.396451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:07:42.396451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:07:42.396451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:07:42.397403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:07:42.397403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:07:42.397403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:07:42.397403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:07:42.397403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:42.397403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:42.397403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:42.397403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:42.397403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:42.397403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:42.397403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:42.397403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:42.397403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:42.397403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:42.397403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:42.397403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:42.398452 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:42.398452 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:42.398452 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:42.398452 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:42.398452 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:07:42.398452 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:07:42.398452 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:07:42.398452 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:07:42.398452 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:07:42.398452 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:07:42.398452 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:07:42.398452 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:42.401488 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641da42-dfae-45db-8a20-a44adc0ea916 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:42.402654 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 371/765] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Aug 5 14:07:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:07:42.406382 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e52a98-3db0-4058-aeed-aec5c1a0063f None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:42.454813 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:42.533776 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641da42-dfae-45db-8a20-a44adc0ea916 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:42.571007 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8795ae1-0895-4999-aa58-4e6d07ea71eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:42.571538 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:42.572118 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8795ae1-0895-4999-aa58-4e6d07ea71eb None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:42.572628 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8795ae1-0895-4999-aa58-4e6d07ea71eb None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:42.573051 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8795ae1-0895-4999-aa58-4e6d07ea71eb None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:42.573903 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8795ae1-0895-4999-aa58-4e6d07ea71eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:42.574354 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8795ae1-0895-4999-aa58-4e6d07ea71eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:42.575963 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8795ae1-0895-4999-aa58-4e6d07ea71eb None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:42.590475 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 395/766] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:42.600197 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8795ae1-0895-4999-aa58-4e6d07ea71eb None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:42.676314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e52a98-3db0-4058-aeed-aec5c1a0063f None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:42.711244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88288d39-cbc3-4b16-a6da-aba8e3bd48ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:42.711866 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88288d39-cbc3-4b16-a6da-aba8e3bd48ef None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:42.712138 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88288d39-cbc3-4b16-a6da-aba8e3bd48ef None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:42.712404 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88288d39-cbc3-4b16-a6da-aba8e3bd48ef None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:42.713131 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88288d39-cbc3-4b16-a6da-aba8e3bd48ef None admin] RBAC: Adding query filter params (name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:42.713426 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88288d39-cbc3-4b16-a6da-aba8e3bd48ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:42.714778 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88288d39-cbc3-4b16-a6da-aba8e3bd48ef None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:42.728229 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 372/767] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Aug 5 14:07:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:42.729572 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88288d39-cbc3-4b16-a6da-aba8e3bd48ef None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:42.776485 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:42.783478 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8795ae1-0895-4999-aa58-4e6d07ea71eb None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:42.823190 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2555fbe6-8c0f-4f07-8128-5221e6e78747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:42.824103 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555fbe6-8c0f-4f07-8128-5221e6e78747 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:42.824545 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555fbe6-8c0f-4f07-8128-5221e6e78747 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:42.824952 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555fbe6-8c0f-4f07-8128-5221e6e78747 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:42.832630 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2555fbe6-8c0f-4f07-8128-5221e6e78747 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:42.835452 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2555fbe6-8c0f-4f07-8128-5221e6e78747 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:42.844385 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88288d39-cbc3-4b16-a6da-aba8e3bd48ef None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:42.847877 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-2555fbe6-8c0f-4f07-8128-5221e6e78747 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:42.847877 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:42.847877 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:42.847877 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:42.847877 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:42.847877 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:42.847877 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:42.847877 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:42.847877 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:42.847877 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:42.847877 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:42.847877 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:42.847877 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 14:07:42.847877 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 14:07:42.847877 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 14:07:42.848941 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 14:07:42.848941 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:42.848941 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:42.848941 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:42.848941 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:42.848941 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:42.848941 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:42.848941 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:42.848941 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:42.848941 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:42.848941 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:42.848941 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:42.848941 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:42.848941 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:42.848941 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:42.848941 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:42.848941 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:42.849897 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 14:07:42.849897 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 14:07:42.849897 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:07:42.849897 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:07:42.849897 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:07:42.849897 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:07:42.849897 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:42.849897 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:42.856220 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 396/768] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:07:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:42.857692 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dab53c-008d-4424-8ce5-02f04089a90a None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:42.858118 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dab53c-008d-4424-8ce5-02f04089a90a None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:42.858554 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dab53c-008d-4424-8ce5-02f04089a90a None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:42.859017 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:42.860614 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 397/769] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:42.861480 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dab53c-008d-4424-8ce5-02f04089a90a None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:42.892404 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ddba70-6351-44cc-8dbe-22b2bd0501a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:42.893346 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ddba70-6351-44cc-8dbe-22b2bd0501a0 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:42.894155 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ddba70-6351-44cc-8dbe-22b2bd0501a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:42.894444 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ddba70-6351-44cc-8dbe-22b2bd0501a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:42.907534 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ddba70-6351-44cc-8dbe-22b2bd0501a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:42.909923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ddba70-6351-44cc-8dbe-22b2bd0501a0 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:42.929154 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-56ddba70-6351-44cc-8dbe-22b2bd0501a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:42.929154 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:42.929154 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:42.929154 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:42.929154 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:42.929154 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:42.929154 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:42.929154 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:42.929154 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:42.929154 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:42.929154 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:42.929154 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:42.929154 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:42.929154 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:42.929154 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:42.930059 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:42.930059 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:42.930059 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:42.930059 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:42.930059 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:42.930059 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:42.930059 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:42.930059 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:42.930059 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:42.930059 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:42.930059 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:42.930059 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:42.930059 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:42.930059 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:42.930059 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:42.930059 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:42.930059 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:42.933268 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:42.933268 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:42.933268 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:42.933268 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:42.933268 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 373/770] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:42.933268 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae6793-5e8e-4d37-9f03-bc039ec1c81c None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:42.933268 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae6793-5e8e-4d37-9f03-bc039ec1c81c None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:42.933268 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae6793-5e8e-4d37-9f03-bc039ec1c81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:42.954218 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-0eae6793-5e8e-4d37-9f03-bc039ec1c81c None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:42.955726 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-0eae6793-5e8e-4d37-9f03-bc039ec1c81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:42.990127 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dab53c-008d-4424-8ce5-02f04089a90a None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:43.031896 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 374/771] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:07:43.033088 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae6793-5e8e-4d37-9f03-bc039ec1c81c None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:43.044113 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314b5a0-2ce5-4e92-87f0-160e0d7ff4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:43.044786 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314b5a0-2ce5-4e92-87f0-160e0d7ff4e6 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:43.045066 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314b5a0-2ce5-4e92-87f0-160e0d7ff4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:43.045339 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314b5a0-2ce5-4e92-87f0-160e0d7ff4e6 None admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:43.046111 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5314b5a0-2ce5-4e92-87f0-160e0d7ff4e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:43.046415 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5314b5a0-2ce5-4e92-87f0-160e0d7ff4e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:43.048671 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5314b5a0-2ce5-4e92-87f0-160e0d7ff4e6 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:43.070906 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 398/772] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Aug 5 14:07:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:43.071808 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314b5a0-2ce5-4e92-87f0-160e0d7ff4e6 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:43.100291 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:43.162416 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:43.168517 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae6793-5e8e-4d37-9f03-bc039ec1c81c None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:43.206577 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207913c3-a89b-463c-bb17-fb6aae297c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:43.207244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207913c3-a89b-463c-bb17-fb6aae297c50 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:43.207525 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207913c3-a89b-463c-bb17-fb6aae297c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:43.207801 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207913c3-a89b-463c-bb17-fb6aae297c50 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:43.208601 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207913c3-a89b-463c-bb17-fb6aae297c50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:43.208910 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207913c3-a89b-463c-bb17-fb6aae297c50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:43.210280 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207913c3-a89b-463c-bb17-fb6aae297c50 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:43.216237 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314b5a0-2ce5-4e92-87f0-160e0d7ff4e6 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:43.231999 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 375/773] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:43.231999 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207913c3-a89b-463c-bb17-fb6aae297c50 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:43.259199 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861f56a-0980-4db8-8352-7eb0a85e1ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:43.259773 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861f56a-0980-4db8-8352-7eb0a85e1ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:43.260054 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861f56a-0980-4db8-8352-7eb0a85e1ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:43.260325 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861f56a-0980-4db8-8352-7eb0a85e1ee1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:43.261180 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1861f56a-0980-4db8-8352-7eb0a85e1ee1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:43.262439 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1861f56a-0980-4db8-8352-7eb0a85e1ee1 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:43.272214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-1861f56a-0980-4db8-8352-7eb0a85e1ee1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 14:07:43.272214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:43.272214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 14:07:43.272214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 14:07:43.272214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 14:07:43.272214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 14:07:43.272214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 14:07:43.272214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:07:43.272214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 14:07:43.272214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:43.272214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 14:07:43.272214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:43.272214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:43.272214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:43.272214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:43.272214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:43.273183 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:43.273183 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:43.273183 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:43.273183 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:43.273183 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:43.273183 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:43.273183 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:43.273183 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 14:07:43.273183 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 14:07:43.273183 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 14:07:43.273183 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 14:07:43.273183 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:43.273183 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:43.273183 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:43.273183 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:43.273183 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:43.274619 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:43.274619 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:43.274619 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:43.274619 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:43.274619 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:43.274619 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:43.274619 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:43.274619 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:43.274619 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:43.274619 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:43.274619 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:43.274619 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 14:07:43.274619 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:07:43.274619 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 14:07:43.274619 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:43.274619 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 399/774] 10.222.0.26 () {56 vars in 1129 bytes} [Fri Aug 5 14:07:43 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:43.275459 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861f56a-0980-4db8-8352-7eb0a85e1ee1 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:43.361646 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207913c3-a89b-463c-bb17-fb6aae297c50 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:43.372616 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:43.391466 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861f56a-0980-4db8-8352-7eb0a85e1ee1 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:43.409926 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289baf47-2ad0-45dc-91bf-f84bf594aac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:43.411269 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289baf47-2ad0-45dc-91bf-f84bf594aac3 None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:43.411768 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289baf47-2ad0-45dc-91bf-f84bf594aac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:43.412236 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289baf47-2ad0-45dc-91bf-f84bf594aac3 None admin] PATH_INFO: `/v3/projects/319973e011c54b4ca1f2198c8a5e135b/users/5ec4940c99fc4fa8a5bc7cdabc4558fb/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:43.414937 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289baf47-2ad0-45dc-91bf-f84bf594aac3 None admin] RBAC: Authorizing `identity:create_grant(project_id=319973e011c54b4ca1f2198c8a5e135b, user_id=5ec4940c99fc4fa8a5bc7cdabc4558fb, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:43.417160 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289baf47-2ad0-45dc-91bf-f84bf594aac3 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:43.429485 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a6626-9d65-4f0c-9bd2-994588bbd9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:43.430232 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a6626-9d65-4f0c-9bd2-994588bbd9f6 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:43.430514 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a6626-9d65-4f0c-9bd2-994588bbd9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:43.430786 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a6626-9d65-4f0c-9bd2-994588bbd9f6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:43.439002 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79a6626-9d65-4f0c-9bd2-994588bbd9f6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:43.440599 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79a6626-9d65-4f0c-9bd2-994588bbd9f6 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:43.452402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-d79a6626-9d65-4f0c-9bd2-994588bbd9f6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 14:07:43.452402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:43.452402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:43.452402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:43.452402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:43.452402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:43.452402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:43.452402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:43.452402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:43.452402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:43.452402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:43.452402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:43.452402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 14:07:43.452402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 14:07:43.452402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 14:07:43.452402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 14:07:43.453365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:43.453365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:43.453365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:07:43.453365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:43.453365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:07:43.453365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:43.453365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:43.453365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:43.453365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:43.453365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:43.453365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:43.453365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:43.453365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:43.453365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:43.453365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:43.453365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:43.453365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:43.453365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:43.454442 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:43.454442 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:43.454442 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 14:07:43.454442 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 14:07:43.454442 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 14:07:43.454442 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 14:07:43.454442 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 14:07:43.454442 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 14:07:43.454442 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 14:07:43.454442 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:43.456559 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 400/775] 10.222.0.26 () {56 vars in 1117 bytes} [Fri Aug 5 14:07:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:43.457563 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a6626-9d65-4f0c-9bd2-994588bbd9f6 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:43.496556 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 376/776] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Aug 5 14:07:43 2022] PUT /identity/v3/projects/319973e011c54b4ca1f2198c8a5e135b/users/5ec4940c99fc4fa8a5bc7cdabc4558fb/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:07:43.497900 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289baf47-2ad0-45dc-91bf-f84bf594aac3 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:43.551345 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:43.586077 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a6626-9d65-4f0c-9bd2-994588bbd9f6 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:43.626806 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97809568-1c91-41d2-be4a-ea4d13a7ccf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:43.627665 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97809568-1c91-41d2-be4a-ea4d13a7ccf3 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:43.628983 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97809568-1c91-41d2-be4a-ea4d13a7ccf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:43.629412 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97809568-1c91-41d2-be4a-ea4d13a7ccf3 None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:43.630315 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97809568-1c91-41d2-be4a-ea4d13a7ccf3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:43.630767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97809568-1c91-41d2-be4a-ea4d13a7ccf3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:43.632423 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97809568-1c91-41d2-be4a-ea4d13a7ccf3 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:43.638640 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 401/777] 10.222.0.26 () {56 vars in 1124 bytes} [Fri Aug 5 14:07:43 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:43.644639 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97809568-1c91-41d2-be4a-ea4d13a7ccf3 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:43.648782 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:43.655967 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289baf47-2ad0-45dc-91bf-f84bf594aac3 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:43.693512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc04d56-78db-48ae-b282-a6c8518515f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:43.694322 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc04d56-78db-48ae-b282-a6c8518515f5 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:43.694694 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc04d56-78db-48ae-b282-a6c8518515f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:43.695063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc04d56-78db-48ae-b282-a6c8518515f5 None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:43.695960 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc04d56-78db-48ae-b282-a6c8518515f5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:43.696365 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc04d56-78db-48ae-b282-a6c8518515f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:43.697937 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc04d56-78db-48ae-b282-a6c8518515f5 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:43.721486 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 377/778] 10.222.0.26 () {56 vars in 1149 bytes} [Fri Aug 5 14:07:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:43.722991 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc04d56-78db-48ae-b282-a6c8518515f5 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:43.761800 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97809568-1c91-41d2-be4a-ea4d13a7ccf3 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:43.799620 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5900513d-5357-424c-8b85-b5f870b46926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:43.801566 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5900513d-5357-424c-8b85-b5f870b46926 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:43.801997 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5900513d-5357-424c-8b85-b5f870b46926 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:43.802418 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5900513d-5357-424c-8b85-b5f870b46926 None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:43.803273 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5900513d-5357-424c-8b85-b5f870b46926 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:43.803726 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5900513d-5357-424c-8b85-b5f870b46926 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:43.805354 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5900513d-5357-424c-8b85-b5f870b46926 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:43.805817 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:43.811467 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 402/779] 10.222.0.26 () {56 vars in 1124 bytes} [Fri Aug 5 14:07:43 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:43.818243 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5900513d-5357-424c-8b85-b5f870b46926 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:43.830925 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc04d56-78db-48ae-b282-a6c8518515f5 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:43.872679 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29507f5-4bd1-4cde-81ca-1f62e70b73cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:43.873414 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29507f5-4bd1-4cde-81ca-1f62e70b73cc None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:43.873756 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29507f5-4bd1-4cde-81ca-1f62e70b73cc None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:43.874089 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29507f5-4bd1-4cde-81ca-1f62e70b73cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:43.887053 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29507f5-4bd1-4cde-81ca-1f62e70b73cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:43.888585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29507f5-4bd1-4cde-81ca-1f62e70b73cc None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:43.897660 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:43.904182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-a29507f5-4bd1-4cde-81ca-1f62e70b73cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:43.904182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:43.904182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:43.904182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:43.904182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:43.904182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:43.904182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:43.904182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:43.904182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:43.904182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:43.904182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:43.904182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:43.904182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:43.904182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:43.904182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:43.905109 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:43.905109 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:43.905109 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:43.905109 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:43.905109 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:43.905109 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:43.905109 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:43.905109 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:43.905109 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:43.905109 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:43.905109 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:43.905109 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:43.905109 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:43.905109 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:43.905109 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:43.905109 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:43.905109 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:43.906017 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:43.906017 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:43.906017 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:43.906017 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:43.911383 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 378/780] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:43.916055 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29507f5-4bd1-4cde-81ca-1f62e70b73cc None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:43.939439 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5900513d-5357-424c-8b85-b5f870b46926 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:43.977141 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb41afe9-1d08-48b2-b5ad-081f33719519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:43.978020 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41afe9-1d08-48b2-b5ad-081f33719519 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:43.978452 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41afe9-1d08-48b2-b5ad-081f33719519 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:43.978929 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41afe9-1d08-48b2-b5ad-081f33719519 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:43.979897 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb41afe9-1d08-48b2-b5ad-081f33719519 None admin] RBAC: Adding query filter params (interface=public, service_id=aacbd040374946769dea3efa4907f83d, region_id=RegionOne) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:43.980358 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb41afe9-1d08-48b2-b5ad-081f33719519 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:43.981875 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb41afe9-1d08-48b2-b5ad-081f33719519 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:43.990889 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 403/781] 10.222.0.26 () {56 vars in 1251 bytes} [Fri Aug 5 14:07:43 2022] GET /identity/v3/endpoints?service_id=aacbd040374946769dea3efa4907f83d&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:43.996824 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb41afe9-1d08-48b2-b5ad-081f33719519 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:44.019307 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29507f5-4bd1-4cde-81ca-1f62e70b73cc None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:44.053803 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8155889f-17a2-4ab0-be1d-3f652c5577ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:44.054568 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155889f-17a2-4ab0-be1d-3f652c5577ce None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:44.054942 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155889f-17a2-4ab0-be1d-3f652c5577ce None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:44.055311 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155889f-17a2-4ab0-be1d-3f652c5577ce None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:44.056097 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8155889f-17a2-4ab0-be1d-3f652c5577ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:44.056847 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8155889f-17a2-4ab0-be1d-3f652c5577ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:44.058027 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8155889f-17a2-4ab0-be1d-3f652c5577ce None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:44.060790 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:44.075471 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 379/782] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:44.077183 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e787662-7ebf-460f-82c3-af6febde2c1d None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:44.077183 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e787662-7ebf-460f-82c3-af6febde2c1d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:44.077755 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e787662-7ebf-460f-82c3-af6febde2c1d None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:44.079780 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 380/783] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:44.080676 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e787662-7ebf-460f-82c3-af6febde2c1d None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:44.113292 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb41afe9-1d08-48b2-b5ad-081f33719519 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:44.145611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:44.151841 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50556414-ac7e-4460-8f36-42436b7ad73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:44.152770 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50556414-ac7e-4460-8f36-42436b7ad73f None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:44.153237 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50556414-ac7e-4460-8f36-42436b7ad73f None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:44.153817 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50556414-ac7e-4460-8f36-42436b7ad73f None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:44.154687 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50556414-ac7e-4460-8f36-42436b7ad73f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:44.156275 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50556414-ac7e-4460-8f36-42436b7ad73f None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:44.168587 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 404/784] 10.222.0.26 () {56 vars in 1087 bytes} [Fri Aug 5 14:07:43 2022] GET /identity/v3/services => generated 772 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:44.170385 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243114c9-ba43-4e0c-8a44-7b844bf32211 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:44.170879 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243114c9-ba43-4e0c-8a44-7b844bf32211 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:44.171446 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243114c9-ba43-4e0c-8a44-7b844bf32211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:44.194475 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e787662-7ebf-460f-82c3-af6febde2c1d None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:44.210350 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-243114c9-ba43-4e0c-8a44-7b844bf32211 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:44.211565 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-243114c9-ba43-4e0c-8a44-7b844bf32211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:44.247934 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8af7f0a-2975-4c2e-a005-05160bb2871d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:44.248629 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af7f0a-2975-4c2e-a005-05160bb2871d None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:44.248984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af7f0a-2975-4c2e-a005-05160bb2871d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:44.249252 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af7f0a-2975-4c2e-a005-05160bb2871d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:44.256816 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8af7f0a-2975-4c2e-a005-05160bb2871d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:44.258321 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8af7f0a-2975-4c2e-a005-05160bb2871d None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:44.272314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-f8af7f0a-2975-4c2e-a005-05160bb2871d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:44.272314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:44.272314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:44.272314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:44.272314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:44.272314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:44.272314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:44.272314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:44.272314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:44.272314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:44.272314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:44.272314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:44.272314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 14:07:44.272314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 14:07:44.272314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 14:07:44.273321 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 14:07:44.273321 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:44.273321 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:44.273321 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:44.273321 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:44.273321 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:44.273321 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:44.273321 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:44.273321 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:44.273321 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:44.273321 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:44.273321 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:44.273321 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:44.273321 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:44.273321 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:44.273321 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:44.273321 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:44.274233 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 14:07:44.274233 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 14:07:44.274233 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:07:44.274233 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:07:44.274233 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:07:44.274233 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:07:44.274233 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:44.274233 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:44.277737 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 381/785] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:07:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:44.278104 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8af7f0a-2975-4c2e-a005-05160bb2871d None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:44.320442 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 405/786] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:07:44.322108 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9925a2-9e07-4b83-8c3b-d2168b2eed7f None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:44.322766 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9925a2-9e07-4b83-8c3b-d2168b2eed7f None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:44.323246 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9925a2-9e07-4b83-8c3b-d2168b2eed7f None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:44.325251 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 406/787] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:44.338010 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99d7f5-0889-47cf-af3d-4fbfa020382d None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:44.338681 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99d7f5-0889-47cf-af3d-4fbfa020382d None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:44.339220 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99d7f5-0889-47cf-af3d-4fbfa020382d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:44.345309 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:44.362590 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-3b99d7f5-0889-47cf-af3d-4fbfa020382d None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:44.363876 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-3b99d7f5-0889-47cf-af3d-4fbfa020382d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:44.415807 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8af7f0a-2975-4c2e-a005-05160bb2871d None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:44.449728 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 407/788] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:07:44.451574 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b99d7f5-0889-47cf-af3d-4fbfa020382d None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:44.457003 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e25fc-0b9f-4f22-abe2-8ebe8097b03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:44.457708 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e25fc-0b9f-4f22-abe2-8ebe8097b03b None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:44.457987 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e25fc-0b9f-4f22-abe2-8ebe8097b03b None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:44.458259 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e25fc-0b9f-4f22-abe2-8ebe8097b03b None admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:44.459072 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35e25fc-0b9f-4f22-abe2-8ebe8097b03b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:44.459381 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35e25fc-0b9f-4f22-abe2-8ebe8097b03b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:44.460952 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35e25fc-0b9f-4f22-abe2-8ebe8097b03b None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:44.478119 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 382/789] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Aug 5 14:07:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:44.485544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e25fc-0b9f-4f22-abe2-8ebe8097b03b None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:44.583590 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b99d7f5-0889-47cf-af3d-4fbfa020382d None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:44.610684 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e25fc-0b9f-4f22-abe2-8ebe8097b03b None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:44.619755 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:44.625580 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:44.631900 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26035ed2-ecb1-48f0-9035-8d1244a34595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:44.632583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26035ed2-ecb1-48f0-9035-8d1244a34595 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:44.632868 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26035ed2-ecb1-48f0-9035-8d1244a34595 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:44.633140 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26035ed2-ecb1-48f0-9035-8d1244a34595 None admin] PATH_INFO: `/v3/services/network` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:44.633776 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26035ed2-ecb1-48f0-9035-8d1244a34595 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:44.635147 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26035ed2-ecb1-48f0-9035-8d1244a34595 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:44.647753 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-26035ed2-ecb1-48f0-9035-8d1244a34595 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 14:07:44.647753 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:44.647753 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 14:07:44.647753 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 14:07:44.647753 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 14:07:44.647753 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 14:07:44.647753 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 14:07:44.647753 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:07:44.647753 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 14:07:44.647753 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:44.647753 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 14:07:44.647753 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:44.647753 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:44.647753 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:44.647753 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:44.647753 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:44.647753 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:44.649612 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:44.649612 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:44.649612 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:44.649612 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:44.649612 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:44.649612 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:44.649612 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 14:07:44.649612 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 14:07:44.649612 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 14:07:44.649612 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 14:07:44.649612 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:44.649612 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:44.649612 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:44.649612 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:44.649612 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:44.649612 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:44.650672 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:44.650672 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:44.650672 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:44.650672 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:44.650672 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:44.650672 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:44.650672 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:44.650672 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:44.650672 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:44.650672 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:44.650672 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 14:07:44.650672 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:07:44.650672 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 14:07:44.650672 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:44.650672 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 408/790] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:07:44 2022] GET /identity/v3/services/network => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:44.651666 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26035ed2-ecb1-48f0-9035-8d1244a34595 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:44.658715 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb00a9f5-c0ff-45b8-ac89-0e1532948df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:44.660106 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00a9f5-c0ff-45b8-ac89-0e1532948df0 None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:44.660558 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00a9f5-c0ff-45b8-ac89-0e1532948df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:44.660962 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00a9f5-c0ff-45b8-ac89-0e1532948df0 None admin] PATH_INFO: `/v3/projects/319973e011c54b4ca1f2198c8a5e135b/users/17e3b509a2de4159bf234594573f5e18/roles/ec18a2850fd540a39d4a935348bb4474` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:44.680212 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb00a9f5-c0ff-45b8-ac89-0e1532948df0 None admin] RBAC: Authorizing `identity:create_grant(project_id=319973e011c54b4ca1f2198c8a5e135b, user_id=17e3b509a2de4159bf234594573f5e18, role_id=ec18a2850fd540a39d4a935348bb4474)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:44.681863 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb00a9f5-c0ff-45b8-ac89-0e1532948df0 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:44.760487 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26035ed2-ecb1-48f0-9035-8d1244a34595 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:44.764576 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 383/791] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Aug 5 14:07:44 2022] PUT /identity/v3/projects/319973e011c54b4ca1f2198c8a5e135b/users/17e3b509a2de4159bf234594573f5e18/roles/ec18a2850fd540a39d4a935348bb4474 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:07:44.766907 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb00a9f5-c0ff-45b8-ac89-0e1532948df0 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:44.806175 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:44.835910 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d86f5f5-e823-4978-9c41-cf8f751934ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:44.836601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86f5f5-e823-4978-9c41-cf8f751934ad None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:44.836880 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86f5f5-e823-4978-9c41-cf8f751934ad None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:44.837147 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86f5f5-e823-4978-9c41-cf8f751934ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:44.863989 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d86f5f5-e823-4978-9c41-cf8f751934ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:07:44.873288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-1d86f5f5-e823-4978-9c41-cf8f751934ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:07:44.873288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:44.873288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:44.873288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:44.873288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:44.873288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:44.873288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:44.873288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:44.873288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:44.873288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:44.873288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:44.873288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:44.873288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:07:44.873288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:07:44.873288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:07:44.873288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:07:44.874323 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:07:44.874323 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:07:44.874323 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:07:44.874323 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:07:44.874323 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:44.874323 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:44.874323 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:44.874323 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:44.874323 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:44.874323 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:44.874323 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:44.874323 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:44.874323 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:44.874323 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:44.874323 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:44.874323 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:44.875167 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:44.875167 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:44.875167 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:44.875167 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:44.875167 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:07:44.875167 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:07:44.875167 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:07:44.875167 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:07:44.875167 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:07:44.875167 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:07:44.875167 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:07:44.875167 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:44.880106 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 409/792] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Aug 5 14:07:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:44.886501 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d86f5f5-e823-4978-9c41-cf8f751934ad None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:45.000328 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb00a9f5-c0ff-45b8-ac89-0e1532948df0 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:45.012724 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d86f5f5-e823-4978-9c41-cf8f751934ad None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:45.049553 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a9fb7-781b-4e2d-95b7-9df1a3b0f58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:45.050951 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a9fb7-781b-4e2d-95b7-9df1a3b0f58e None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:45.051552 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a9fb7-781b-4e2d-95b7-9df1a3b0f58e None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:45.054774 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a9fb7-781b-4e2d-95b7-9df1a3b0f58e None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:45.054774 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a9fb7-781b-4e2d-95b7-9df1a3b0f58e None admin] RBAC: Adding query filter params (name=network) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:45.054774 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a9fb7-781b-4e2d-95b7-9df1a3b0f58e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:45.054774 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a9fb7-781b-4e2d-95b7-9df1a3b0f58e None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:45.060904 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 384/793] 10.222.0.26 () {56 vars in 1112 bytes} [Fri Aug 5 14:07:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:45.067264 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a9fb7-781b-4e2d-95b7-9df1a3b0f58e None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:45.073566 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6142f0-8374-437b-b5ce-b88b9ab25ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:45.074201 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6142f0-8374-437b-b5ce-b88b9ab25ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:45.074476 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6142f0-8374-437b-b5ce-b88b9ab25ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:45.074744 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6142f0-8374-437b-b5ce-b88b9ab25ee7 None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:45.075461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6142f0-8374-437b-b5ce-b88b9ab25ee7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:45.075763 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6142f0-8374-437b-b5ce-b88b9ab25ee7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:45.080450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6142f0-8374-437b-b5ce-b88b9ab25ee7 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:45.087844 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 410/794] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:07:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:45.093277 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6142f0-8374-437b-b5ce-b88b9ab25ee7 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:45.125321 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:45.145383 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:45.204347 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6142f0-8374-437b-b5ce-b88b9ab25ee7 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:45.253765 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e5ba1-5e0e-4fb0-90f8-7d4ce188db20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:45.254650 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e5ba1-5e0e-4fb0-90f8-7d4ce188db20 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:45.257793 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e5ba1-5e0e-4fb0-90f8-7d4ce188db20 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:45.260758 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e5ba1-5e0e-4fb0-90f8-7d4ce188db20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:45.284428 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613e5ba1-5e0e-4fb0-90f8-7d4ce188db20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:45.287007 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613e5ba1-5e0e-4fb0-90f8-7d4ce188db20 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:45.298612 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a9fb7-781b-4e2d-95b7-9df1a3b0f58e None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:45.329049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-613e5ba1-5e0e-4fb0-90f8-7d4ce188db20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:45.329049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:45.329049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:45.329049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:45.329049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:45.329049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:45.329049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:45.329049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:45.329049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:45.329049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:45.329049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:45.329049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:45.329049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:45.329049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:45.329049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:45.330196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:45.330196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:45.330196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:45.330196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:45.330196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:45.330196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:45.330196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:45.330196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:45.330196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:45.330196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:45.330196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:45.330196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:45.330196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:45.330196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:45.330196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:45.330196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:45.330196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:45.331764 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:45.331764 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:45.331764 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:45.331764 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:45.339662 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 411/795] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:07:45.349559 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e5ba1-5e0e-4fb0-90f8-7d4ce188db20 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:45.403298 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953a82b-078e-4701-96a1-ed510cdc1423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:45.404217 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953a82b-078e-4701-96a1-ed510cdc1423 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:45.404734 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953a82b-078e-4701-96a1-ed510cdc1423 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:45.405183 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953a82b-078e-4701-96a1-ed510cdc1423 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:45.406041 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953a82b-078e-4701-96a1-ed510cdc1423 None admin] RBAC: Adding query filter params (type=network) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:45.406508 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953a82b-078e-4701-96a1-ed510cdc1423 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:45.414957 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953a82b-078e-4701-96a1-ed510cdc1423 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:45.427658 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 385/796] 10.222.0.26 () {56 vars in 1112 bytes} [Fri Aug 5 14:07:45 2022] GET /identity/v3/services?type=network => generated 350 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:45.437613 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953a82b-078e-4701-96a1-ed510cdc1423 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:45.642819 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:45.652279 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e5ba1-5e0e-4fb0-90f8-7d4ce188db20 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:45.695025 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:45.730120 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d83be11-7cbb-46da-8a61-840016531fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:45.730757 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83be11-7cbb-46da-8a61-840016531fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:45.731036 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83be11-7cbb-46da-8a61-840016531fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:45.731585 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83be11-7cbb-46da-8a61-840016531fb7 None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:45.732005 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d83be11-7cbb-46da-8a61-840016531fb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:45.732309 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d83be11-7cbb-46da-8a61-840016531fb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:45.733185 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953a82b-078e-4701-96a1-ed510cdc1423 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:45.733791 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d83be11-7cbb-46da-8a61-840016531fb7 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:45.761816 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 412/797] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:45.765510 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d83be11-7cbb-46da-8a61-840016531fb7 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:45.806377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb05e45-7486-4884-8496-47fcfa884054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:45.807369 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb05e45-7486-4884-8496-47fcfa884054 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:45.807878 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb05e45-7486-4884-8496-47fcfa884054 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:45.810726 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb05e45-7486-4884-8496-47fcfa884054 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:45.811739 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb05e45-7486-4884-8496-47fcfa884054 None admin] RBAC: Adding query filter params (interface=public, service_id=a7b78a5282a3457194842ceb734eaefd, region_id=RegionOne) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:45.812258 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb05e45-7486-4884-8496-47fcfa884054 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:45.813897 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb05e45-7486-4884-8496-47fcfa884054 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:45.830411 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 386/798] 10.222.0.26 () {56 vars in 1251 bytes} [Fri Aug 5 14:07:45 2022] GET /identity/v3/endpoints?service_id=a7b78a5282a3457194842ceb734eaefd&interface=public®ion_id=RegionOne => generated 194 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:45.835278 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb05e45-7486-4884-8496-47fcfa884054 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:45.918592 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d83be11-7cbb-46da-8a61-840016531fb7 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:45.990107 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d21731-90e0-480c-bef5-09875160cf13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:45.991130 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d21731-90e0-480c-bef5-09875160cf13 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:45.991649 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d21731-90e0-480c-bef5-09875160cf13 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:45.992135 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d21731-90e0-480c-bef5-09875160cf13 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:46.009698 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d21731-90e0-480c-bef5-09875160cf13 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:46.011441 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d21731-90e0-480c-bef5-09875160cf13 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:46.023185 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb05e45-7486-4884-8496-47fcfa884054 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:46.024443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-88d21731-90e0-480c-bef5-09875160cf13 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 14:07:46.024443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:46.024443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:46.024443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:46.024443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:46.024443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:46.024443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:46.024443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:46.024443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:46.024443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:46.024443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:46.024443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:46.024443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 14:07:46.024443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 14:07:46.024443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 14:07:46.024443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 14:07:46.025365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:46.025365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:46.025365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:07:46.025365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:46.025365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:07:46.025365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:46.025365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:46.025365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:46.025365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:46.025365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:46.025365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:46.025365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:46.025365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:46.025365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:46.025365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:46.025365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:46.025365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:46.025365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:46.026303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:46.026303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:46.026303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 14:07:46.026303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 14:07:46.026303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 14:07:46.026303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 14:07:46.026303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 14:07:46.026303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 14:07:46.026303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 14:07:46.026303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:46.028869 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 413/799] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Aug 5 14:07:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:46.030304 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e364f46-dc39-4d56-b798-01785b413c0c None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:46.030803 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e364f46-dc39-4d56-b798-01785b413c0c None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:46.031331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e364f46-dc39-4d56-b798-01785b413c0c None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:46.036869 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 414/800] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:46.037960 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e364f46-dc39-4d56-b798-01785b413c0c None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:46.055154 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:46.067127 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f1ad5b-70b5-4620-a88e-ea7ef8e1dc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:46.067626 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1ad5b-70b5-4620-a88e-ea7ef8e1dc75 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:46.067881 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1ad5b-70b5-4620-a88e-ea7ef8e1dc75 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:46.068048 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1ad5b-70b5-4620-a88e-ea7ef8e1dc75 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:46.068713 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f1ad5b-70b5-4620-a88e-ea7ef8e1dc75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:46.070070 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f1ad5b-70b5-4620-a88e-ea7ef8e1dc75 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:46.075812 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 387/801] 10.222.0.26 () {56 vars in 1087 bytes} [Fri Aug 5 14:07:45 2022] GET /identity/v3/services => generated 772 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:46.083313 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845aed4-1a55-43dd-9aa9-61db6db7036d None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:46.084523 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845aed4-1a55-43dd-9aa9-61db6db7036d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:46.088705 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845aed4-1a55-43dd-9aa9-61db6db7036d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:46.132874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-a845aed4-1a55-43dd-9aa9-61db6db7036d None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:46.134786 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a845aed4-1a55-43dd-9aa9-61db6db7036d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:46.145540 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:46.225843 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 388/802] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:07:46.246494 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e364f46-dc39-4d56-b798-01785b413c0c None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:46.249156 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845aed4-1a55-43dd-9aa9-61db6db7036d None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:46.312246 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff30e714-9ee0-40f6-a2db-194777409dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:46.313446 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30e714-9ee0-40f6-a2db-194777409dcd None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:46.313446 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30e714-9ee0-40f6-a2db-194777409dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:46.313645 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30e714-9ee0-40f6-a2db-194777409dcd None admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:46.314410 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff30e714-9ee0-40f6-a2db-194777409dcd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:46.314634 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff30e714-9ee0-40f6-a2db-194777409dcd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:46.317714 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff30e714-9ee0-40f6-a2db-194777409dcd None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:46.341199 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 415/803] 10.222.0.26 () {56 vars in 1135 bytes} [Fri Aug 5 14:07:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:46.345372 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff30e714-9ee0-40f6-a2db-194777409dcd None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:46.382632 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845aed4-1a55-43dd-9aa9-61db6db7036d None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:46.452803 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:46.460133 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d304d79-0faf-47e0-afe5-0c2ef0572cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:46.460966 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d304d79-0faf-47e0-afe5-0c2ef0572cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:46.461348 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d304d79-0faf-47e0-afe5-0c2ef0572cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:46.461722 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d304d79-0faf-47e0-afe5-0c2ef0572cd8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:46.475392 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d304d79-0faf-47e0-afe5-0c2ef0572cd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:07:46.480779 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-6d304d79-0faf-47e0-afe5-0c2ef0572cd8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:07:46.480779 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:46.480779 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:46.480779 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:46.480779 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:46.480779 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:46.480779 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:46.480779 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:46.480779 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:46.480779 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:46.480779 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:46.480779 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:46.480779 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:07:46.480779 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:07:46.480779 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:07:46.480779 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:07:46.481900 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:07:46.481900 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:07:46.481900 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:07:46.481900 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:07:46.481900 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:46.481900 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:46.481900 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:46.481900 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:46.481900 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:46.481900 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:46.481900 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:46.481900 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:46.481900 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:46.481900 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:46.481900 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:46.481900 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:46.482923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:46.482923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:46.482923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:46.482923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:46.482923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:07:46.482923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:07:46.482923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:07:46.482923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:07:46.482923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:07:46.482923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:07:46.482923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:07:46.482923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:46.483713 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff30e714-9ee0-40f6-a2db-194777409dcd None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:46.490596 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 389/804] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Aug 5 14:07:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:07:46.492041 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed63ad2-b4c1-407a-9670-2b4de4fc902a None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:46.492468 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed63ad2-b4c1-407a-9670-2b4de4fc902a None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:46.492849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed63ad2-b4c1-407a-9670-2b4de4fc902a None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:46.494675 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 390/805] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:46.496172 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed63ad2-b4c1-407a-9670-2b4de4fc902a None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:46.504273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:46.519870 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dd08e2-5dce-4496-9581-b082cfc51291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:46.520234 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd08e2-5dce-4496-9581-b082cfc51291 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:46.523255 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd08e2-5dce-4496-9581-b082cfc51291 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:46.523255 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd08e2-5dce-4496-9581-b082cfc51291 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:46.537025 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dd08e2-5dce-4496-9581-b082cfc51291 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:46.537025 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dd08e2-5dce-4496-9581-b082cfc51291 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:46.557134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-80dd08e2-5dce-4496-9581-b082cfc51291 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:46.557134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:46.557134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:46.557134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:46.557134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:46.557134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:46.557134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:46.557134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:46.557134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:46.557134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:46.557134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:46.557134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:46.557134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:46.557134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:46.557134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:46.558257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:46.558257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:46.558257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:46.558257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:46.558257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:46.558257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:46.558257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:46.558257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:46.558257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:46.558257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:46.558257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:46.558257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:46.558257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:46.558257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:46.558257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:46.558257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:46.558257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:46.559278 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:46.559278 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:46.559278 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:46.559278 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:46.559278 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 416/806] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:46.560450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7d54f-d77f-4c7f-a895-17517dfa8744 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:46.561153 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7d54f-d77f-4c7f-a895-17517dfa8744 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:46.561504 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7d54f-d77f-4c7f-a895-17517dfa8744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:46.587157 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-8ec7d54f-d77f-4c7f-a895-17517dfa8744 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:46.588575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-8ec7d54f-d77f-4c7f-a895-17517dfa8744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:46.612040 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed63ad2-b4c1-407a-9670-2b4de4fc902a None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:46.656735 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f2674-4b47-43cc-b4cb-4070412f436a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:46.657524 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f2674-4b47-43cc-b4cb-4070412f436a None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:46.657875 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f2674-4b47-43cc-b4cb-4070412f436a None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:46.658268 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f2674-4b47-43cc-b4cb-4070412f436a None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:46.659742 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f2674-4b47-43cc-b4cb-4070412f436a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:46.660062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f2674-4b47-43cc-b4cb-4070412f436a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:46.662161 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f2674-4b47-43cc-b4cb-4070412f436a None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:46.674151 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 391/808] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:07:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:46.675728 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f2674-4b47-43cc-b4cb-4070412f436a None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:46.679070 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 417/808] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:07:46.680395 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7d54f-d77f-4c7f-a895-17517dfa8744 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:46.756600 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:46.757051 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:46.801347 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f2674-4b47-43cc-b4cb-4070412f436a None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:46.812020 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7d54f-d77f-4c7f-a895-17517dfa8744 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:46.846161 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c059b4-b1e2-43c5-b52d-1d05ea3de19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:46.847238 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c059b4-b1e2-43c5-b52d-1d05ea3de19d None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:46.847521 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c059b4-b1e2-43c5-b52d-1d05ea3de19d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:46.847831 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c059b4-b1e2-43c5-b52d-1d05ea3de19d None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:46.848664 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c059b4-b1e2-43c5-b52d-1d05ea3de19d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:46.848971 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c059b4-b1e2-43c5-b52d-1d05ea3de19d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:46.850328 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c059b4-b1e2-43c5-b52d-1d05ea3de19d None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:46.850691 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00468e11-202e-4ff1-918e-6dd688149ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:46.851392 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00468e11-202e-4ff1-918e-6dd688149ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:46.851850 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00468e11-202e-4ff1-918e-6dd688149ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:46.852184 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00468e11-202e-4ff1-918e-6dd688149ca3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:46.863761 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00468e11-202e-4ff1-918e-6dd688149ca3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:46.865346 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00468e11-202e-4ff1-918e-6dd688149ca3 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:46.865517 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 392/809] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:46.866282 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c059b4-b1e2-43c5-b52d-1d05ea3de19d None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:46.882935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-00468e11-202e-4ff1-918e-6dd688149ca3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:46.882935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:46.882935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:46.882935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:46.882935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:46.882935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:46.882935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:46.882935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:46.882935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:46.882935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:46.882935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:46.882935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:46.882935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:46.882935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:46.882935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:46.885003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:46.885003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:46.885003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:46.885003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:46.885003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:46.885003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:46.885003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:46.885003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:46.885003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:46.885003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:46.885003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:46.885003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:46.885003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:46.885003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:46.885003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:46.885003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:46.885003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:46.887075 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:46.887075 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:46.887075 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:46.887075 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:46.892436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 418/810] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:46.892436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00468e11-202e-4ff1-918e-6dd688149ca3 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:46.994756 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00468e11-202e-4ff1-918e-6dd688149ca3 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:47.001332 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c059b4-b1e2-43c5-b52d-1d05ea3de19d None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:47.002161 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:47.033049 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:47.040882 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66644cae-2a93-4c9c-aa7b-fb06cc798320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:47.041907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66644cae-2a93-4c9c-aa7b-fb06cc798320 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:47.042459 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66644cae-2a93-4c9c-aa7b-fb06cc798320 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:47.042884 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66644cae-2a93-4c9c-aa7b-fb06cc798320 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:47.045513 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec3432-e0c9-4d5e-b2fb-10e9ee12cbd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:47.049560 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec3432-e0c9-4d5e-b2fb-10e9ee12cbd6 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:47.049560 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec3432-e0c9-4d5e-b2fb-10e9ee12cbd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:47.049560 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec3432-e0c9-4d5e-b2fb-10e9ee12cbd6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:47.049560 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eec3432-e0c9-4d5e-b2fb-10e9ee12cbd6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:47.049560 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eec3432-e0c9-4d5e-b2fb-10e9ee12cbd6 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:47.051584 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66644cae-2a93-4c9c-aa7b-fb06cc798320 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:47.053359 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66644cae-2a93-4c9c-aa7b-fb06cc798320 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:47.062350 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-7eec3432-e0c9-4d5e-b2fb-10e9ee12cbd6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 14:07:47.062350 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:47.062350 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 14:07:47.062350 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 14:07:47.062350 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 14:07:47.062350 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 14:07:47.062350 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 14:07:47.062350 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:07:47.062350 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 14:07:47.062350 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:47.062350 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 14:07:47.062350 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:47.062350 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:47.062350 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:47.062350 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:47.062350 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:47.063405 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:47.063405 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:47.063405 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:47.063405 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:47.063405 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:47.063405 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:47.063405 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:47.063405 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 14:07:47.063405 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 14:07:47.063405 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 14:07:47.063405 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 14:07:47.063405 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:47.063405 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:47.063405 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:47.063405 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:47.063405 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:47.064262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:47.064262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:47.064262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:47.064262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:47.064262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:47.064262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:47.064262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:47.064262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:47.064262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:47.064262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:47.064262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:47.064262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 14:07:47.064262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:07:47.064262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 14:07:47.064262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:47.064262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 393/811] 10.222.0.26 () {56 vars in 1129 bytes} [Fri Aug 5 14:07:46 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:47.065158 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec3432-e0c9-4d5e-b2fb-10e9ee12cbd6 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:47.071024 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-66644cae-2a93-4c9c-aa7b-fb06cc798320 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:47.071024 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:47.071024 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:47.071024 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:47.071024 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:47.071024 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:47.071024 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:47.071024 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:47.071024 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:47.071024 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:47.071024 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:47.071024 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:47.071024 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 14:07:47.071024 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 14:07:47.071024 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 14:07:47.072052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 14:07:47.072052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:47.072052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:47.072052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:47.072052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:47.072052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:47.072052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:47.072052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:47.072052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:47.072052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:47.072052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:47.072052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:47.072052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:47.072052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:47.072052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:47.072052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:47.072052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:47.072980 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 14:07:47.072980 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 14:07:47.072980 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:07:47.072980 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:07:47.072980 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:07:47.072980 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:07:47.072980 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:47.072980 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:47.080233 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 419/812] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:07:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:47.081396 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66644cae-2a93-4c9c-aa7b-fb06cc798320 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:47.200048 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66644cae-2a93-4c9c-aa7b-fb06cc798320 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:47.202026 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec3432-e0c9-4d5e-b2fb-10e9ee12cbd6 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:47.238990 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0905084-9e62-43b9-83c7-c3a2c57fbfd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:47.239698 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0905084-9e62-43b9-83c7-c3a2c57fbfd4 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:47.240046 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0905084-9e62-43b9-83c7-c3a2c57fbfd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:47.240449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0905084-9e62-43b9-83c7-c3a2c57fbfd4 None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:47.241327 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0905084-9e62-43b9-83c7-c3a2c57fbfd4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:47.241702 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0905084-9e62-43b9-83c7-c3a2c57fbfd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:47.243223 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0905084-9e62-43b9-83c7-c3a2c57fbfd4 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:47.248454 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370c781-a6e5-4c56-91d2-8d3969904999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:47.248844 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 420/813] 10.222.0.26 () {56 vars in 1124 bytes} [Fri Aug 5 14:07:47 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:47.249571 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370c781-a6e5-4c56-91d2-8d3969904999 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:47.249827 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0905084-9e62-43b9-83c7-c3a2c57fbfd4 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:47.250216 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370c781-a6e5-4c56-91d2-8d3969904999 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:47.250684 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370c781-a6e5-4c56-91d2-8d3969904999 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:47.251544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f370c781-a6e5-4c56-91d2-8d3969904999 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:47.252032 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f370c781-a6e5-4c56-91d2-8d3969904999 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:47.255065 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f370c781-a6e5-4c56-91d2-8d3969904999 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:47.271627 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 394/814] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:47.272739 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370c781-a6e5-4c56-91d2-8d3969904999 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:47.284234 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:47.338099 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:47.367871 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0905084-9e62-43b9-83c7-c3a2c57fbfd4 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:47.397068 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370c781-a6e5-4c56-91d2-8d3969904999 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:47.411164 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3f5da-809b-4017-9bd4-fb476e271ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:47.411784 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3f5da-809b-4017-9bd4-fb476e271ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:47.412065 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3f5da-809b-4017-9bd4-fb476e271ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:47.412337 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3f5da-809b-4017-9bd4-fb476e271ba2 None admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:47.413143 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b3f5da-809b-4017-9bd4-fb476e271ba2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:47.413450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b3f5da-809b-4017-9bd4-fb476e271ba2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:47.414869 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b3f5da-809b-4017-9bd4-fb476e271ba2 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:47.427736 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 421/815] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Aug 5 14:07:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:47.428371 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3f5da-809b-4017-9bd4-fb476e271ba2 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:47.438541 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f1b1eb-c5db-419a-b6a8-ea9c3517e13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:47.439166 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1b1eb-c5db-419a-b6a8-ea9c3517e13a None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:47.439445 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1b1eb-c5db-419a-b6a8-ea9c3517e13a None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:47.440018 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1b1eb-c5db-419a-b6a8-ea9c3517e13a None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:47.440424 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f1b1eb-c5db-419a-b6a8-ea9c3517e13a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:47.440960 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f1b1eb-c5db-419a-b6a8-ea9c3517e13a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:47.442417 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f1b1eb-c5db-419a-b6a8-ea9c3517e13a None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:47.453333 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 395/816] 10.222.0.26 () {56 vars in 1124 bytes} [Fri Aug 5 14:07:47 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:47.454182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe79b08-a173-41ed-810a-6ef745a083a3 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:47.454464 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe79b08-a173-41ed-810a-6ef745a083a3 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:47.454740 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe79b08-a173-41ed-810a-6ef745a083a3 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:47.457261 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 396/817] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:47.457437 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe79b08-a173-41ed-810a-6ef745a083a3 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:47.531533 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:47.542403 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3f5da-809b-4017-9bd4-fb476e271ba2 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:47.574466 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe79b08-a173-41ed-810a-6ef745a083a3 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:47.579670 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76fe9a5-58d3-4afa-8b2c-f8ee24684bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:47.580312 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76fe9a5-58d3-4afa-8b2c-f8ee24684bd1 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:47.580616 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76fe9a5-58d3-4afa-8b2c-f8ee24684bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:47.581421 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76fe9a5-58d3-4afa-8b2c-f8ee24684bd1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:47.589247 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76fe9a5-58d3-4afa-8b2c-f8ee24684bd1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:47.591044 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76fe9a5-58d3-4afa-8b2c-f8ee24684bd1 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:47.592181 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:47.603393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-f76fe9a5-58d3-4afa-8b2c-f8ee24684bd1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 14:07:47.603393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:47.603393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:47.603393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:47.603393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:47.603393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:47.603393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:47.603393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:47.603393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:47.603393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:47.603393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:47.603393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:47.603393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 14:07:47.603393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 14:07:47.603393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 14:07:47.603393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 14:07:47.609798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:47.609798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:47.609798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:07:47.609798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:47.609798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:07:47.609798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:47.609798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:47.609798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:47.609798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:47.609798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:47.609798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:47.609798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:47.609798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:47.609798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:47.609798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:47.609798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:47.609798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:47.609798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:47.610747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:47.610747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:47.610747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 14:07:47.610747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 14:07:47.610747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 14:07:47.610747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 14:07:47.610747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 14:07:47.610747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 14:07:47.610747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 14:07:47.610747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:47.610747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 422/818] 10.222.0.26 () {56 vars in 1093 bytes} [Fri Aug 5 14:07:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:47.610747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76fe9a5-58d3-4afa-8b2c-f8ee24684bd1 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:47.624147 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27895cd3-90de-42cf-8a9d-482879b3210e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:47.626019 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27895cd3-90de-42cf-8a9d-482879b3210e None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:47.626019 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27895cd3-90de-42cf-8a9d-482879b3210e None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:47.626019 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27895cd3-90de-42cf-8a9d-482879b3210e None admin] PATH_INFO: `/v3/projects/319973e011c54b4ca1f2198c8a5e135b/users/dddd7eea67f34f0cbb5f6a1cd85f1fc5/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:47.628799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27895cd3-90de-42cf-8a9d-482879b3210e None admin] RBAC: Authorizing `identity:create_grant(project_id=319973e011c54b4ca1f2198c8a5e135b, user_id=dddd7eea67f34f0cbb5f6a1cd85f1fc5, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:47.630750 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27895cd3-90de-42cf-8a9d-482879b3210e None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:47.708015 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76fe9a5-58d3-4afa-8b2c-f8ee24684bd1 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:47.716849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 397/819] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Aug 5 14:07:47 2022] PUT /identity/v3/projects/319973e011c54b4ca1f2198c8a5e135b/users/dddd7eea67f34f0cbb5f6a1cd85f1fc5/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:07:47.718656 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94ddfa-cc9d-4daf-bde6-0bf4b91f8096 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:47.718944 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94ddfa-cc9d-4daf-bde6-0bf4b91f8096 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:47.719225 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94ddfa-cc9d-4daf-bde6-0bf4b91f8096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:47.755056 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-1c94ddfa-cc9d-4daf-bde6-0bf4b91f8096 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:47.755946 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7800c-8a80-4374-94fd-9ec3753e8929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:47.756382 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-1c94ddfa-cc9d-4daf-bde6-0bf4b91f8096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:47.757214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7800c-8a80-4374-94fd-9ec3753e8929 None admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:47.757721 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7800c-8a80-4374-94fd-9ec3753e8929 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:47.758222 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7800c-8a80-4374-94fd-9ec3753e8929 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:47.759354 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc7800c-8a80-4374-94fd-9ec3753e8929 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:47.761171 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc7800c-8a80-4374-94fd-9ec3753e8929 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:47.840440 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 398/820] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:07:47.840440 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94ddfa-cc9d-4daf-bde6-0bf4b91f8096 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:47.877025 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 423/821] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:07:47 2022] POST /identity/v3/endpoints => generated 346 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:07:47.879306 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7800c-8a80-4374-94fd-9ec3753e8929 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:47.890692 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:47.978200 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94ddfa-cc9d-4daf-bde6-0bf4b91f8096 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:47.981015 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:48.014921 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7800c-8a80-4374-94fd-9ec3753e8929 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:48.031375 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e895e306-3ca3-4b9e-84db-2df962dd0260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:48.032373 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895e306-3ca3-4b9e-84db-2df962dd0260 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:48.033029 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895e306-3ca3-4b9e-84db-2df962dd0260 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:48.034232 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895e306-3ca3-4b9e-84db-2df962dd0260 None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:48.035291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e895e306-3ca3-4b9e-84db-2df962dd0260 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:48.035828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e895e306-3ca3-4b9e-84db-2df962dd0260 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:48.038170 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e895e306-3ca3-4b9e-84db-2df962dd0260 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:48.061896 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 399/822] 10.222.0.26 () {56 vars in 1133 bytes} [Fri Aug 5 14:07:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:48.067383 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e895e306-3ca3-4b9e-84db-2df962dd0260 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:48.095031 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ed521-8467-49f4-b2b7-42ef4648a771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:48.096383 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ed521-8467-49f4-b2b7-42ef4648a771 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:48.097426 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ed521-8467-49f4-b2b7-42ef4648a771 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:48.099013 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ed521-8467-49f4-b2b7-42ef4648a771 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:48.121428 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573ed521-8467-49f4-b2b7-42ef4648a771 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:07:48.127008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-573ed521-8467-49f4-b2b7-42ef4648a771 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:07:48.127008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:48.127008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:48.127008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:48.127008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:48.127008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:48.127008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:48.127008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:48.127008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:48.127008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:48.127008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:48.127008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:48.127008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:07:48.127008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:07:48.127008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:07:48.127008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:07:48.128243 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:07:48.128243 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:07:48.128243 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:07:48.128243 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:07:48.128243 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:48.128243 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:48.128243 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:48.128243 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:48.128243 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:48.128243 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:48.128243 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:48.128243 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:48.128243 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:48.128243 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:48.128243 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:48.128243 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:48.129763 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:48.129763 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:48.129763 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:48.129763 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:48.129763 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:07:48.129763 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:07:48.129763 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:07:48.129763 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:07:48.129763 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:07:48.129763 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:07:48.129763 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:07:48.129763 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:48.129763 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 424/823] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Aug 5 14:07:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:48.134948 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ed521-8467-49f4-b2b7-42ef4648a771 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:48.209073 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:48.235990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e895e306-3ca3-4b9e-84db-2df962dd0260 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:48.249811 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ed521-8467-49f4-b2b7-42ef4648a771 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:48.287870 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d5407-20e0-4f1e-91e7-ea95f099a4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:48.289050 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d5407-20e0-4f1e-91e7-ea95f099a4fa None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:48.289615 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d5407-20e0-4f1e-91e7-ea95f099a4fa None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:48.290117 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d5407-20e0-4f1e-91e7-ea95f099a4fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:48.291224 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2295c36-c5e6-4327-ba38-ee4d2ebff9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:48.291948 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2295c36-c5e6-4327-ba38-ee4d2ebff9cc None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:48.292219 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2295c36-c5e6-4327-ba38-ee4d2ebff9cc None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:48.292512 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2295c36-c5e6-4327-ba38-ee4d2ebff9cc None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:48.293226 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2295c36-c5e6-4327-ba38-ee4d2ebff9cc None admin] RBAC: Adding query filter params (name=service) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:48.293521 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2295c36-c5e6-4327-ba38-ee4d2ebff9cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:48.294875 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2295c36-c5e6-4327-ba38-ee4d2ebff9cc None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:48.304820 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 425/824] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Aug 5 14:07:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:48.310225 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2295c36-c5e6-4327-ba38-ee4d2ebff9cc None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:48.333546 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:48.389862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d5407-20e0-4f1e-91e7-ea95f099a4fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:48.391772 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d5407-20e0-4f1e-91e7-ea95f099a4fa None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:48.454502 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2295c36-c5e6-4327-ba38-ee4d2ebff9cc None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:48.480857 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:48.493256 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465e95f-c122-47e1-aacf-17e374c39c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:48.494452 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465e95f-c122-47e1-aacf-17e374c39c24 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:48.494988 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465e95f-c122-47e1-aacf-17e374c39c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:48.495727 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465e95f-c122-47e1-aacf-17e374c39c24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:48.507718 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6465e95f-c122-47e1-aacf-17e374c39c24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:48.509721 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6465e95f-c122-47e1-aacf-17e374c39c24 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:48.513870 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 400/825] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:07:48 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 14:07:48.517550 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f8e67-35ed-4879-9e61-f9afab96b139 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:48.517550 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f8e67-35ed-4879-9e61-f9afab96b139 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:48.517550 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f8e67-35ed-4879-9e61-f9afab96b139 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:48.525487 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 401/826] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:48.526104 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f8e67-35ed-4879-9e61-f9afab96b139 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:48.541277 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-6465e95f-c122-47e1-aacf-17e374c39c24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:48.541277 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:48.541277 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:48.541277 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:48.541277 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:48.541277 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:48.541277 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:48.541277 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:48.541277 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:48.541277 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:48.541277 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:48.541277 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:48.541277 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:48.541277 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:48.541277 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:48.542191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:48.542191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:48.542191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:48.542191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:48.542191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:48.542191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:48.542191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:48.542191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:48.542191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:48.542191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:48.542191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:48.542191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:48.542191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:48.542191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:48.542191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:48.542191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:48.542191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:48.543096 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:48.543096 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:48.543096 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:48.543096 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:48.545439 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 426/827] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:07:48.546934 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114bc20-eaea-4922-92e7-2f07217ad3cd None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:48.547412 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114bc20-eaea-4922-92e7-2f07217ad3cd None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:48.547879 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114bc20-eaea-4922-92e7-2f07217ad3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:48.571252 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-9114bc20-eaea-4922-92e7-2f07217ad3cd None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:48.572887 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-9114bc20-eaea-4922-92e7-2f07217ad3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:48.608576 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:48.660763 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 427/828] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:07:48.661741 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9114bc20-eaea-4922-92e7-2f07217ad3cd None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:48.663814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f8e67-35ed-4879-9e61-f9afab96b139 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:48.707939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61f8b6-c204-4de8-9df0-42628b1cb3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:48.709215 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61f8b6-c204-4de8-9df0-42628b1cb3a9 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:48.709693 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61f8b6-c204-4de8-9df0-42628b1cb3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:48.710178 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61f8b6-c204-4de8-9df0-42628b1cb3a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:48.725956 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c61f8b6-c204-4de8-9df0-42628b1cb3a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:48.728321 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c61f8b6-c204-4de8-9df0-42628b1cb3a9 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:48.751455 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-6c61f8b6-c204-4de8-9df0-42628b1cb3a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:48.751455 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:48.751455 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:48.751455 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:48.751455 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:48.751455 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:48.751455 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:48.751455 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:48.751455 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:48.751455 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:48.751455 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:48.751455 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:48.751455 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:48.751455 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:48.751455 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:48.752339 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:48.752339 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:48.752339 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:48.752339 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:48.752339 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:48.752339 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:48.752339 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:48.752339 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:48.752339 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:48.752339 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:48.752339 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:48.752339 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:48.752339 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:48.752339 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:48.752339 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:48.752339 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:48.752339 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:48.753314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:48.753314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:48.753314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:48.753314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:48.756343 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 402/829] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:48.757464 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61f8b6-c204-4de8-9df0-42628b1cb3a9 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:48.789496 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9114bc20-eaea-4922-92e7-2f07217ad3cd None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:48.800836 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:48.840609 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89a713-1942-48ad-946c-fc5defc9d817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:48.841743 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89a713-1942-48ad-946c-fc5defc9d817 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:48.842168 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89a713-1942-48ad-946c-fc5defc9d817 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:48.842813 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89a713-1942-48ad-946c-fc5defc9d817 None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:48.844593 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e89a713-1942-48ad-946c-fc5defc9d817 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:48.845047 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e89a713-1942-48ad-946c-fc5defc9d817 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:48.847321 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e89a713-1942-48ad-946c-fc5defc9d817 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:48.864024 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 428/830] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:48.865196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89a713-1942-48ad-946c-fc5defc9d817 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:48.888642 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61f8b6-c204-4de8-9df0-42628b1cb3a9 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:48.907645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:48.939049 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9958e0b-80bf-41dd-bb0d-0aaaf29cc407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:48.940191 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9958e0b-80bf-41dd-bb0d-0aaaf29cc407 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:48.941217 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9958e0b-80bf-41dd-bb0d-0aaaf29cc407 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:48.941658 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9958e0b-80bf-41dd-bb0d-0aaaf29cc407 None admin] PATH_INFO: `/v3/services/network` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:48.943127 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9958e0b-80bf-41dd-bb0d-0aaaf29cc407 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:48.944966 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9958e0b-80bf-41dd-bb0d-0aaaf29cc407 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:48.957697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-b9958e0b-80bf-41dd-bb0d-0aaaf29cc407 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 14:07:48.957697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:48.957697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 14:07:48.957697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 14:07:48.957697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 14:07:48.957697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 14:07:48.957697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 14:07:48.957697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:07:48.957697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 14:07:48.957697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:48.957697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 14:07:48.957697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:48.957697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:48.957697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:48.957697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:48.957697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:48.957697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:48.958889 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:48.958889 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:48.958889 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:48.958889 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:48.958889 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:48.958889 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:48.958889 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 14:07:48.958889 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 14:07:48.958889 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 14:07:48.958889 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 14:07:48.958889 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:48.958889 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:48.958889 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:48.958889 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:48.958889 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:48.958889 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:48.959909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:48.959909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:48.959909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:48.959909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:48.959909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:48.959909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:48.959909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:48.959909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:48.959909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:48.959909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:48.959909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 14:07:48.959909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:07:48.959909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 14:07:48.959909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:48.967120 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 403/831] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:07:48 2022] GET /identity/v3/services/network => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:48.967120 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9958e0b-80bf-41dd-bb0d-0aaaf29cc407 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:48.993461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89a713-1942-48ad-946c-fc5defc9d817 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:49.039457 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48cfc0a-a0c0-4345-8ff4-9ec07201665e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:49.040922 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48cfc0a-a0c0-4345-8ff4-9ec07201665e None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:49.042156 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48cfc0a-a0c0-4345-8ff4-9ec07201665e None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:49.042584 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48cfc0a-a0c0-4345-8ff4-9ec07201665e None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:49.043424 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48cfc0a-a0c0-4345-8ff4-9ec07201665e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:49.043893 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48cfc0a-a0c0-4345-8ff4-9ec07201665e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:49.045459 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48cfc0a-a0c0-4345-8ff4-9ec07201665e None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:49.062220 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 429/832] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:49.063346 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48cfc0a-a0c0-4345-8ff4-9ec07201665e None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:49.078937 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9958e0b-80bf-41dd-bb0d-0aaaf29cc407 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:49.093628 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:49.125365 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6804d-fd05-4d49-8be7-f5ba45b1ffcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:49.126888 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6804d-fd05-4d49-8be7-f5ba45b1ffcf None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:49.127309 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6804d-fd05-4d49-8be7-f5ba45b1ffcf None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:49.128233 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6804d-fd05-4d49-8be7-f5ba45b1ffcf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:49.138644 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff6804d-fd05-4d49-8be7-f5ba45b1ffcf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:49.140771 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff6804d-fd05-4d49-8be7-f5ba45b1ffcf None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:49.157303 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-eff6804d-fd05-4d49-8be7-f5ba45b1ffcf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 14:07:49.157303 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:49.157303 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:49.157303 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:49.157303 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:49.157303 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:49.157303 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:49.157303 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:49.157303 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:49.157303 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:49.157303 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:49.157303 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:49.157303 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 14:07:49.157303 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 14:07:49.157303 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 14:07:49.157303 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 14:07:49.158478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:49.158478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:49.158478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:07:49.158478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:49.158478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:07:49.158478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:49.158478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:49.158478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:49.158478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:49.158478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:49.158478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:49.158478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:49.158478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:49.158478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:49.158478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:49.158478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:49.158478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:49.158478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:49.159779 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:49.159779 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:49.159779 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 14:07:49.159779 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 14:07:49.159779 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 14:07:49.159779 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 14:07:49.159779 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 14:07:49.159779 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 14:07:49.159779 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 14:07:49.159779 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:49.162038 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 404/833] 10.222.0.26 () {56 vars in 1117 bytes} [Fri Aug 5 14:07:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:49.163031 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6804d-fd05-4d49-8be7-f5ba45b1ffcf None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:49.177142 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48cfc0a-a0c0-4345-8ff4-9ec07201665e None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:49.205382 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:49.220665 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e6b6b0-e791-4566-b079-ed84a6270d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:49.221510 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e6b6b0-e791-4566-b079-ed84a6270d33 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:49.221957 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e6b6b0-e791-4566-b079-ed84a6270d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:49.222535 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e6b6b0-e791-4566-b079-ed84a6270d33 None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:49.223651 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e6b6b0-e791-4566-b079-ed84a6270d33 None admin] RBAC: Adding query filter params (name=network) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:49.224097 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e6b6b0-e791-4566-b079-ed84a6270d33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:49.225622 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e6b6b0-e791-4566-b079-ed84a6270d33 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:49.236798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 430/834] 10.222.0.26 () {56 vars in 1112 bytes} [Fri Aug 5 14:07:49 2022] GET /identity/v3/services?name=network => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:49.238010 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e6b6b0-e791-4566-b079-ed84a6270d33 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:49.279111 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6804d-fd05-4d49-8be7-f5ba45b1ffcf None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:49.317823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026193cb-bfe7-4075-96a0-e67cdde52ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:49.318545 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026193cb-bfe7-4075-96a0-e67cdde52ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:49.318910 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026193cb-bfe7-4075-96a0-e67cdde52ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:49.319574 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026193cb-bfe7-4075-96a0-e67cdde52ec0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:49.326734 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026193cb-bfe7-4075-96a0-e67cdde52ec0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:49.328109 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026193cb-bfe7-4075-96a0-e67cdde52ec0 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:49.336635 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:49.344273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-026193cb-bfe7-4075-96a0-e67cdde52ec0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:49.344273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:49.344273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:49.344273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:49.344273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:49.344273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:49.344273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:49.344273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:49.344273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:49.344273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:49.344273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:49.344273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:49.344273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 14:07:49.344273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 14:07:49.344273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 14:07:49.349727 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 14:07:49.349727 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:49.349727 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:49.349727 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:49.349727 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:49.349727 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:49.349727 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:49.349727 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:49.349727 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:49.349727 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:49.349727 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:49.349727 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:49.349727 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:49.349727 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:49.349727 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:49.349727 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:49.349727 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:49.350622 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 14:07:49.350622 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 14:07:49.350622 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:07:49.350622 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:07:49.350622 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:07:49.350622 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:07:49.350622 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:49.350622 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:49.350622 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 405/835] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:07:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:49.351100 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e6b6b0-e791-4566-b079-ed84a6270d33 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:49.351174 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026193cb-bfe7-4075-96a0-e67cdde52ec0 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:49.386038 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eddbf8-51b7-41c8-8ab4-d0c80bff9355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:49.386905 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eddbf8-51b7-41c8-8ab4-d0c80bff9355 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:49.387368 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eddbf8-51b7-41c8-8ab4-d0c80bff9355 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:49.387836 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eddbf8-51b7-41c8-8ab4-d0c80bff9355 None admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:49.388831 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eddbf8-51b7-41c8-8ab4-d0c80bff9355 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:49.389311 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eddbf8-51b7-41c8-8ab4-d0c80bff9355 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:49.390893 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eddbf8-51b7-41c8-8ab4-d0c80bff9355 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:49.410200 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 431/836] 10.222.0.26 () {56 vars in 1149 bytes} [Fri Aug 5 14:07:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:49.411423 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eddbf8-51b7-41c8-8ab4-d0c80bff9355 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:49.458449 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:49.464991 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026193cb-bfe7-4075-96a0-e67cdde52ec0 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:49.500695 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010cf7a0-7a86-435b-811e-3cd58e43d5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:49.501313 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010cf7a0-7a86-435b-811e-3cd58e43d5b7 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:49.501590 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010cf7a0-7a86-435b-811e-3cd58e43d5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:49.501858 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010cf7a0-7a86-435b-811e-3cd58e43d5b7 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:49.502591 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010cf7a0-7a86-435b-811e-3cd58e43d5b7 None admin] RBAC: Adding query filter params (type=network) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:49.503105 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010cf7a0-7a86-435b-811e-3cd58e43d5b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:49.504568 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010cf7a0-7a86-435b-811e-3cd58e43d5b7 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:49.514133 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 406/837] 10.222.0.26 () {56 vars in 1112 bytes} [Fri Aug 5 14:07:49 2022] GET /identity/v3/services?type=network => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:49.514133 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010cf7a0-7a86-435b-811e-3cd58e43d5b7 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:49.534668 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eddbf8-51b7-41c8-8ab4-d0c80bff9355 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:49.571492 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc868e4e-badb-42ef-a758-a2f2e4ab022c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:49.572468 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc868e4e-badb-42ef-a758-a2f2e4ab022c None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:49.572929 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc868e4e-badb-42ef-a758-a2f2e4ab022c None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:49.573387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc868e4e-badb-42ef-a758-a2f2e4ab022c None admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:49.574303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc868e4e-badb-42ef-a758-a2f2e4ab022c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:49.574779 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc868e4e-badb-42ef-a758-a2f2e4ab022c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:49.576651 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc868e4e-badb-42ef-a758-a2f2e4ab022c None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:49.590456 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 432/838] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Aug 5 14:07:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:49.591535 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc868e4e-badb-42ef-a758-a2f2e4ab022c None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:49.603989 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:49.628609 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010cf7a0-7a86-435b-811e-3cd58e43d5b7 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:49.666959 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2dc0f0-2aa2-4327-b2d1-c1ccbd573dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:49.667692 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2dc0f0-2aa2-4327-b2d1-c1ccbd573dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:49.668062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2dc0f0-2aa2-4327-b2d1-c1ccbd573dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:49.668378 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2dc0f0-2aa2-4327-b2d1-c1ccbd573dd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:49.697613 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:49.704170 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc868e4e-badb-42ef-a758-a2f2e4ab022c None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:49.740604 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2dc0f0-2aa2-4327-b2d1-c1ccbd573dd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:49.742280 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2dc0f0-2aa2-4327-b2d1-c1ccbd573dd9 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:49.742571 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612101e-d689-4576-b0ee-908e3ee9a943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:49.743425 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612101e-d689-4576-b0ee-908e3ee9a943 None admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:49.743917 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612101e-d689-4576-b0ee-908e3ee9a943 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:49.744552 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612101e-d689-4576-b0ee-908e3ee9a943 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:49.745822 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a612101e-d689-4576-b0ee-908e3ee9a943 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:49.747416 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a612101e-d689-4576-b0ee-908e3ee9a943 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:49.846208 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 407/840] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:07:49 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 14:07:49.847134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 433/840] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:07:49 2022] POST /identity/v3/endpoints => generated 337 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:07:49.847845 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2dc0f0-2aa2-4327-b2d1-c1ccbd573dd9 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:49.851918 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612101e-d689-4576-b0ee-908e3ee9a943 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:49.970744 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:49.990387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:50.009977 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2dc0f0-2aa2-4327-b2d1-c1ccbd573dd9 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:50.052168 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612101e-d689-4576-b0ee-908e3ee9a943 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:50.054184 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c4b6f-b473-43f0-8fe4-9ca4e113fc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:50.055364 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c4b6f-b473-43f0-8fe4-9ca4e113fc03 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:50.055364 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c4b6f-b473-43f0-8fe4-9ca4e113fc03 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:50.055364 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c4b6f-b473-43f0-8fe4-9ca4e113fc03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:50.056083 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36c4b6f-b473-43f0-8fe4-9ca4e113fc03 None admin] RBAC: Adding query filter params (role.id=299c794c7acb4a5ca36288c3191aad10, scope.project.id=319973e011c54b4ca1f2198c8a5e135b, user.id=5ec4940c99fc4fa8a5bc7cdabc4558fb) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:50.056392 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36c4b6f-b473-43f0-8fe4-9ca4e113fc03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:50.057852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36c4b6f-b473-43f0-8fe4-9ca4e113fc03 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:50.065653 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 408/841] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:07:49 2022] GET /identity/v3/role_assignments?user.id=5ec4940c99fc4fa8a5bc7cdabc4558fb&scope.project.id=319973e011c54b4ca1f2198c8a5e135b&role.id=299c794c7acb4a5ca36288c3191aad10 => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:50.067190 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f55a06-0834-481c-a995-4228f2485767 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:50.067473 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f55a06-0834-481c-a995-4228f2485767 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:50.067747 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f55a06-0834-481c-a995-4228f2485767 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:50.073347 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 409/842] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:50.084281 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e1771-899e-4664-8568-fef6770b99bb None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:50.084663 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e1771-899e-4664-8568-fef6770b99bb None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:50.084875 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e1771-899e-4664-8568-fef6770b99bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:50.097034 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb04d99c-9fa9-4298-95ad-f92882e5d9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:50.098061 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04d99c-9fa9-4298-95ad-f92882e5d9ba None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:50.098549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04d99c-9fa9-4298-95ad-f92882e5d9ba None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:50.099141 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04d99c-9fa9-4298-95ad-f92882e5d9ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:50.105927 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-d63e1771-899e-4664-8568-fef6770b99bb None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:50.107134 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d63e1771-899e-4664-8568-fef6770b99bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:50.113345 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb04d99c-9fa9-4298-95ad-f92882e5d9ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:50.115154 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb04d99c-9fa9-4298-95ad-f92882e5d9ba None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:50.133052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-cb04d99c-9fa9-4298-95ad-f92882e5d9ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:50.133052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:50.133052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:50.133052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:50.133052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:50.133052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:50.133052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:50.133052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:50.133052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:50.133052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:50.133052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:50.133052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:50.133052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:50.133052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:50.133052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:50.134010 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:50.134010 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:50.134010 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:50.134010 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:50.134010 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:50.134010 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:50.134010 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:50.134010 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:50.134010 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:50.134010 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:50.134010 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:50.134010 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:50.134010 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:50.134010 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:50.134010 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:50.134010 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:50.134010 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:50.134919 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:50.134919 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:50.134919 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:50.134919 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:50.137436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 434/843] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:07:50.144154 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb04d99c-9fa9-4298-95ad-f92882e5d9ba None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:50.193164 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 410/844] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:50 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 14:07:50.218003 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63e1771-899e-4664-8568-fef6770b99bb None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:50.265038 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb04d99c-9fa9-4298-95ad-f92882e5d9ba None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:50.280162 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:50.282376 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:50.307523 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3152f9-c304-49ef-b8fe-6fb4dd82ff33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:50.308148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3152f9-c304-49ef-b8fe-6fb4dd82ff33 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:50.308460 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3152f9-c304-49ef-b8fe-6fb4dd82ff33 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:50.308724 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3152f9-c304-49ef-b8fe-6fb4dd82ff33 None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:50.309419 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3152f9-c304-49ef-b8fe-6fb4dd82ff33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:50.309723 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3152f9-c304-49ef-b8fe-6fb4dd82ff33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:50.311075 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3152f9-c304-49ef-b8fe-6fb4dd82ff33 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:50.323110 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 435/845] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:50.324171 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6b3b7-5de7-452d-baca-90fea48e6e18 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:50.324478 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6b3b7-5de7-452d-baca-90fea48e6e18 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:50.324757 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6b3b7-5de7-452d-baca-90fea48e6e18 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:50.325951 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 436/846] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:50.328139 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6b3b7-5de7-452d-baca-90fea48e6e18 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:50.340636 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63e1771-899e-4664-8568-fef6770b99bb None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:50.380774 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42123191-72f0-41c1-99a7-0a038433d75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:50.381551 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42123191-72f0-41c1-99a7-0a038433d75d None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:50.381889 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42123191-72f0-41c1-99a7-0a038433d75d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:50.382164 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42123191-72f0-41c1-99a7-0a038433d75d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:50.393666 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42123191-72f0-41c1-99a7-0a038433d75d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:07:50.398575 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-42123191-72f0-41c1-99a7-0a038433d75d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:07:50.398575 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:50.398575 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:50.398575 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:50.398575 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:50.398575 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:50.398575 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:50.398575 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:50.398575 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:50.398575 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:50.398575 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:50.398575 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:50.398575 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:07:50.398575 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:07:50.398575 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:07:50.398575 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:07:50.399512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:07:50.399512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:07:50.399512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:07:50.399512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:07:50.399512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:50.399512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:50.399512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:50.399512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:50.399512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:50.399512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:50.399512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:50.399512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:50.399512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:50.399512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:50.399512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:50.399512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:50.400352 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:50.400352 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:50.400352 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:50.400352 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:50.400352 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:07:50.400352 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:07:50.400352 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:07:50.400352 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:07:50.400352 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:07:50.400352 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:07:50.400352 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:07:50.400352 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:50.400352 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 411/847] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Aug 5 14:07:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:07:50.401693 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3189628-2f68-437c-aed1-a88b2670041c None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:50.401693 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3189628-2f68-437c-aed1-a88b2670041c None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:50.401693 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3189628-2f68-437c-aed1-a88b2670041c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:50.419765 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-d3189628-2f68-437c-aed1-a88b2670041c None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:50.420981 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d3189628-2f68-437c-aed1-a88b2670041c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:50.446732 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6b3b7-5de7-452d-baca-90fea48e6e18 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:50.484209 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf1041a-c65b-40b7-98f7-bbf306452e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:50.484890 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf1041a-c65b-40b7-98f7-bbf306452e94 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:50.485166 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf1041a-c65b-40b7-98f7-bbf306452e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:50.485437 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf1041a-c65b-40b7-98f7-bbf306452e94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:50.500434 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 412/848] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:50 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:07:50.500434 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3189628-2f68-437c-aed1-a88b2670041c None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:50.502535 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf1041a-c65b-40b7-98f7-bbf306452e94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:50.505190 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf1041a-c65b-40b7-98f7-bbf306452e94 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:50.519938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-dbf1041a-c65b-40b7-98f7-bbf306452e94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:50.519938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:50.519938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:50.519938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:50.519938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:50.519938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:50.519938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:50.519938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:50.519938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:50.519938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:50.519938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:50.519938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:50.519938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 14:07:50.519938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 14:07:50.519938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 14:07:50.520923 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 14:07:50.520923 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:50.520923 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:50.520923 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:50.520923 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:50.520923 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:50.520923 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:50.520923 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:50.520923 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:50.520923 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:50.520923 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:50.520923 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:50.520923 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:50.520923 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:50.520923 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:50.520923 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:50.520923 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:50.521812 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 14:07:50.521812 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 14:07:50.521812 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:07:50.521812 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:07:50.521812 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:07:50.521812 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:07:50.521812 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:50.521812 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:50.526123 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 437/849] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:07:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:50.527406 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf1041a-c65b-40b7-98f7-bbf306452e94 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:50.547706 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:50.562391 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:50.609381 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3189628-2f68-437c-aed1-a88b2670041c None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:50.635301 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf1041a-c65b-40b7-98f7-bbf306452e94 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:50.647550 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abacb83a-a48f-4b47-a88a-2bc96f5198f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:50.648196 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abacb83a-a48f-4b47-a88a-2bc96f5198f1 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:50.648492 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abacb83a-a48f-4b47-a88a-2bc96f5198f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:50.648762 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abacb83a-a48f-4b47-a88a-2bc96f5198f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:50.649464 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abacb83a-a48f-4b47-a88a-2bc96f5198f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:50.649760 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abacb83a-a48f-4b47-a88a-2bc96f5198f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:50.651115 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abacb83a-a48f-4b47-a88a-2bc96f5198f1 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:50.664998 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 413/850] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:07:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:50.664998 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abacb83a-a48f-4b47-a88a-2bc96f5198f1 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:50.678982 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c30d02c-f69a-4be8-8ff3-5c4ea4a6be91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:50.679627 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30d02c-f69a-4be8-8ff3-5c4ea4a6be91 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:50.679901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30d02c-f69a-4be8-8ff3-5c4ea4a6be91 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:50.680165 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30d02c-f69a-4be8-8ff3-5c4ea4a6be91 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:50.680821 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c30d02c-f69a-4be8-8ff3-5c4ea4a6be91 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:50.682248 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c30d02c-f69a-4be8-8ff3-5c4ea4a6be91 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:50.692121 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-2c30d02c-f69a-4be8-8ff3-5c4ea4a6be91 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 14:07:50.692121 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:50.692121 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 14:07:50.692121 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 14:07:50.692121 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 14:07:50.692121 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 14:07:50.692121 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 14:07:50.692121 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:07:50.692121 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 14:07:50.692121 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:50.692121 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 14:07:50.692121 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:50.692121 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:50.692121 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:50.692121 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:50.692121 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:50.692121 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:50.693289 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:50.693289 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:50.693289 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:50.693289 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:50.693289 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:50.693289 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:50.693289 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 14:07:50.693289 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 14:07:50.693289 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 14:07:50.693289 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 14:07:50.693289 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:50.693289 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:50.693289 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:50.693289 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:50.693289 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:50.693289 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:50.694148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:50.694148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:50.694148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:50.694148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:50.694148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:50.694148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:50.694148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:50.694148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:50.694148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:50.694148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:50.694148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 14:07:50.694148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:07:50.694148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 14:07:50.694148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:50.694148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 438/851] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Aug 5 14:07:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:50.694945 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c30d02c-f69a-4be8-8ff3-5c4ea4a6be91 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:50.778315 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abacb83a-a48f-4b47-a88a-2bc96f5198f1 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:50.779948 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:50.797673 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:50.803527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c30d02c-f69a-4be8-8ff3-5c4ea4a6be91 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:50.814683 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8408b0c9-b89c-4b0a-8f06-ee0777f16e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:50.816144 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8408b0c9-b89c-4b0a-8f06-ee0777f16e9c None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:50.816444 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8408b0c9-b89c-4b0a-8f06-ee0777f16e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:50.816730 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8408b0c9-b89c-4b0a-8f06-ee0777f16e9c None admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:50.817487 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8408b0c9-b89c-4b0a-8f06-ee0777f16e9c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:50.817793 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8408b0c9-b89c-4b0a-8f06-ee0777f16e9c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:50.819225 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8408b0c9-b89c-4b0a-8f06-ee0777f16e9c None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:50.832646 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 414/852] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Aug 5 14:07:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:50.833671 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8408b0c9-b89c-4b0a-8f06-ee0777f16e9c None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:50.842825 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d318e-a973-4c1a-b552-c2a1765c866a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:50.843731 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d318e-a973-4c1a-b552-c2a1765c866a None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:50.844186 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d318e-a973-4c1a-b552-c2a1765c866a None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:50.844487 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d318e-a973-4c1a-b552-c2a1765c866a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:50.857871 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829d318e-a973-4c1a-b552-c2a1765c866a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:50.859622 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829d318e-a973-4c1a-b552-c2a1765c866a None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:50.875076 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-829d318e-a973-4c1a-b552-c2a1765c866a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:50.875076 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:50.875076 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:50.875076 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:50.875076 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:50.875076 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:50.875076 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:50.875076 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:50.875076 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:50.875076 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:50.875076 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:50.875076 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:50.875076 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:50.875076 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:50.875076 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:50.876002 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:50.876002 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:50.876002 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:50.876002 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:50.876002 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:50.876002 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:50.876002 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:50.876002 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:50.876002 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:50.876002 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:50.876002 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:50.876002 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:50.876002 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:50.876002 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:50.876002 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:50.876002 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:50.876002 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:50.877450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:50.877450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:50.877450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:50.877450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:50.880819 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 439/853] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:50.881851 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d318e-a973-4c1a-b552-c2a1765c866a None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:50.941827 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8408b0c9-b89c-4b0a-8f06-ee0777f16e9c None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:50.978511 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7430e41d-518f-4b53-85b9-4ce13b3d7900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:50.979125 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430e41d-518f-4b53-85b9-4ce13b3d7900 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:50.979403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430e41d-518f-4b53-85b9-4ce13b3d7900 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:50.979676 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430e41d-518f-4b53-85b9-4ce13b3d7900 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:50.980347 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7430e41d-518f-4b53-85b9-4ce13b3d7900 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:50.980676 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7430e41d-518f-4b53-85b9-4ce13b3d7900 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:50.982045 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7430e41d-518f-4b53-85b9-4ce13b3d7900 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:50.986469 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 415/854] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Aug 5 14:07:50 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:50.987796 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7430e41d-518f-4b53-85b9-4ce13b3d7900 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:50.995026 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d318e-a973-4c1a-b552-c2a1765c866a None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:51.017659 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:51.034747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b53f42-73bc-4e2e-a61a-939b019dc049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:51.035365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b53f42-73bc-4e2e-a61a-939b019dc049 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:51.035644 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b53f42-73bc-4e2e-a61a-939b019dc049 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:51.035972 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b53f42-73bc-4e2e-a61a-939b019dc049 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:51.036786 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b53f42-73bc-4e2e-a61a-939b019dc049 None admin] RBAC: Adding query filter params (role.id=ec18a2850fd540a39d4a935348bb4474, scope.project.id=319973e011c54b4ca1f2198c8a5e135b, user.id=17e3b509a2de4159bf234594573f5e18) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:51.037144 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b53f42-73bc-4e2e-a61a-939b019dc049 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:51.038717 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b53f42-73bc-4e2e-a61a-939b019dc049 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:51.052455 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 440/855] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:07:50 2022] GET /identity/v3/role_assignments?user.id=17e3b509a2de4159bf234594573f5e18&scope.project.id=319973e011c54b4ca1f2198c8a5e135b&role.id=ec18a2850fd540a39d4a935348bb4474 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:51.052455 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b53f42-73bc-4e2e-a61a-939b019dc049 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:51.070992 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:51.107586 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7430e41d-518f-4b53-85b9-4ce13b3d7900 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:51.152101 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5753c5b9-7926-4419-b39f-e183bbc342e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:51.153266 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753c5b9-7926-4419-b39f-e183bbc342e0 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:51.153646 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753c5b9-7926-4419-b39f-e183bbc342e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:51.154020 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753c5b9-7926-4419-b39f-e183bbc342e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:51.154827 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5753c5b9-7926-4419-b39f-e183bbc342e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:51.155235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5753c5b9-7926-4419-b39f-e183bbc342e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:51.156859 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5753c5b9-7926-4419-b39f-e183bbc342e0 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:51.175542 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 416/856] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:51.176688 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5753c5b9-7926-4419-b39f-e183bbc342e0 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:51.182112 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b53f42-73bc-4e2e-a61a-939b019dc049 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:51.228993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7f48b5-7ff7-47e3-b892-7df693496a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:51.230387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7f48b5-7ff7-47e3-b892-7df693496a41 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:51.230811 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7f48b5-7ff7-47e3-b892-7df693496a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:51.231636 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7f48b5-7ff7-47e3-b892-7df693496a41 None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:51.232768 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7f48b5-7ff7-47e3-b892-7df693496a41 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:51.233281 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7f48b5-7ff7-47e3-b892-7df693496a41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:51.235070 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7f48b5-7ff7-47e3-b892-7df693496a41 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:51.249124 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 441/857] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Aug 5 14:07:51 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:51.294066 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:51.311693 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5753c5b9-7926-4419-b39f-e183bbc342e0 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:51.352879 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795849f-2f6f-4733-90f7-0bf469612ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:51.353675 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795849f-2f6f-4733-90f7-0bf469612ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:51.354048 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795849f-2f6f-4733-90f7-0bf469612ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:51.354417 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795849f-2f6f-4733-90f7-0bf469612ed2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:51.363392 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f795849f-2f6f-4733-90f7-0bf469612ed2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:51.365042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f795849f-2f6f-4733-90f7-0bf469612ed2 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:51.379825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-f795849f-2f6f-4733-90f7-0bf469612ed2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 14:07:51.379825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:51.379825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:51.379825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:51.379825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:51.379825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:51.379825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:51.379825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:51.379825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:51.379825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:51.379825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:51.379825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:51.379825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 14:07:51.379825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 14:07:51.379825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 14:07:51.379825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 14:07:51.381070 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:51.381070 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:51.381070 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:07:51.381070 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:51.381070 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:07:51.381070 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:51.381070 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:51.381070 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:51.381070 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:51.381070 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:51.381070 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:51.381070 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:51.381070 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:51.381070 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:51.381070 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:51.381070 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:51.381070 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:51.381070 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:51.382237 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:51.382237 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:51.382237 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 14:07:51.382237 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 14:07:51.382237 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 14:07:51.382237 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 14:07:51.382237 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 14:07:51.382237 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 14:07:51.382237 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 14:07:51.382237 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:51.383826 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 417/858] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Aug 5 14:07:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:51.389395 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7f48b5-7ff7-47e3-b892-7df693496a41 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:51.479949 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:51.504540 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7f48b5-7ff7-47e3-b892-7df693496a41 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:51.539663 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e8f6e-bd06-40db-8327-8a7b568decb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:51.540280 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e8f6e-bd06-40db-8327-8a7b568decb7 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:51.540663 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e8f6e-bd06-40db-8327-8a7b568decb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:51.540934 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e8f6e-bd06-40db-8327-8a7b568decb7 None admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:51.541731 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271e8f6e-bd06-40db-8327-8a7b568decb7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:51.542036 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271e8f6e-bd06-40db-8327-8a7b568decb7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:51.543452 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271e8f6e-bd06-40db-8327-8a7b568decb7 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:51.561511 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 442/859] 10.222.0.26 () {56 vars in 1135 bytes} [Fri Aug 5 14:07:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:51.567598 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795849f-2f6f-4733-90f7-0bf469612ed2 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:51.677453 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795849f-2f6f-4733-90f7-0bf469612ed2 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:51.716438 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601bcff7-8729-4676-8b2c-50fa43467fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:51.716438 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601bcff7-8729-4676-8b2c-50fa43467fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:51.716438 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601bcff7-8729-4676-8b2c-50fa43467fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:51.716438 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601bcff7-8729-4676-8b2c-50fa43467fa9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:51.732309 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:51.787426 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601bcff7-8729-4676-8b2c-50fa43467fa9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:51.788972 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601bcff7-8729-4676-8b2c-50fa43467fa9 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:51.890085 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 418/860] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:07:51 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 14:07:51.896615 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e8f6e-bd06-40db-8327-8a7b568decb7 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:52.001486 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e8f6e-bd06-40db-8327-8a7b568decb7 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:52.037912 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ae1e1-e736-4cad-8ef3-2d1689464158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:52.038559 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ae1e1-e736-4cad-8ef3-2d1689464158 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:52.038835 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ae1e1-e736-4cad-8ef3-2d1689464158 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:52.039102 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ae1e1-e736-4cad-8ef3-2d1689464158 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:52.041101 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:52.050802 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54ae1e1-e736-4cad-8ef3-2d1689464158 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:52.052256 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54ae1e1-e736-4cad-8ef3-2d1689464158 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:52.067595 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-e54ae1e1-e736-4cad-8ef3-2d1689464158 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:52.067595 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:52.067595 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:52.067595 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:52.067595 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:52.067595 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:52.067595 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:52.067595 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:52.067595 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:52.067595 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:52.067595 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:52.067595 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:52.067595 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:52.067595 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:52.067595 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:52.069043 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:52.069043 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:52.069043 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:52.069043 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:52.069043 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:52.069043 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:52.069043 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:52.069043 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:52.069043 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:52.069043 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:52.069043 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:52.069043 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:52.069043 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:52.069043 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:52.069043 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:52.069043 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:52.069043 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:52.069928 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:52.069928 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:52.069928 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:52.069928 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:52.069928 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 443/861] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:52.074807 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601bcff7-8729-4676-8b2c-50fa43467fa9 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:52.121906 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:52.174963 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601bcff7-8729-4676-8b2c-50fa43467fa9 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:52.192848 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70230f58-0a36-4d3c-b7f6-d89e1819863b None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:52.193133 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70230f58-0a36-4d3c-b7f6-d89e1819863b None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:52.193405 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70230f58-0a36-4d3c-b7f6-d89e1819863b None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:52.195010 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 444/862] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:07:52.199714 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567ad7b-4f41-47ad-8b2c-e46dda2dcaa8 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:52.199994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567ad7b-4f41-47ad-8b2c-e46dda2dcaa8 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:52.200273 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567ad7b-4f41-47ad-8b2c-e46dda2dcaa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:52.212728 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1cb4d9-1ecd-4f37-a54a-dd655ba5ed88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:52.213908 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1cb4d9-1ecd-4f37-a54a-dd655ba5ed88 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:52.214337 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1cb4d9-1ecd-4f37-a54a-dd655ba5ed88 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:52.214772 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1cb4d9-1ecd-4f37-a54a-dd655ba5ed88 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:52.215618 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1cb4d9-1ecd-4f37-a54a-dd655ba5ed88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:52.216068 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1cb4d9-1ecd-4f37-a54a-dd655ba5ed88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:52.217584 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1cb4d9-1ecd-4f37-a54a-dd655ba5ed88 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:52.218724 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-4567ad7b-4f41-47ad-8b2c-e46dda2dcaa8 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:52.220218 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-4567ad7b-4f41-47ad-8b2c-e46dda2dcaa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:52.231429 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 419/863] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:52.237834 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1cb4d9-1ecd-4f37-a54a-dd655ba5ed88 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:52.300598 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 445/864] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:52 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 14:07:52.316267 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567ad7b-4f41-47ad-8b2c-e46dda2dcaa8 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:52.351359 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1cb4d9-1ecd-4f37-a54a-dd655ba5ed88 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:52.360630 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:52.401170 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d459b-4e1d-4987-8482-2c26b319b204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:52.402550 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d459b-4e1d-4987-8482-2c26b319b204 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:52.403060 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d459b-4e1d-4987-8482-2c26b319b204 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:52.403885 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d459b-4e1d-4987-8482-2c26b319b204 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:52.413618 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d459b-4e1d-4987-8482-2c26b319b204 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:52.415794 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d459b-4e1d-4987-8482-2c26b319b204 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:52.429183 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567ad7b-4f41-47ad-8b2c-e46dda2dcaa8 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:52.429814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-027d459b-4e1d-4987-8482-2c26b319b204 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:52.429814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:52.429814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:52.429814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:52.429814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:52.429814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:52.429814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:52.429814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:52.429814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:52.429814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:52.429814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:52.429814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:52.429814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 14:07:52.429814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 14:07:52.429814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 14:07:52.430702 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 14:07:52.430702 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:52.430702 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:52.430702 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:52.430702 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:52.430702 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:52.430702 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:52.430702 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:52.430702 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:52.430702 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:52.430702 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:52.430702 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:52.430702 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:52.430702 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:52.430702 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:52.430702 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:52.430702 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:52.431585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 14:07:52.431585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 14:07:52.431585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:07:52.431585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:07:52.431585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:07:52.431585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:07:52.431585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:52.431585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:52.438575 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 420/865] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:07:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:07:52.439669 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d459b-4e1d-4987-8482-2c26b319b204 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:52.450655 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:52.472502 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d9011-f97d-4a2d-a5d9-b85ca4f0377a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:52.473293 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d9011-f97d-4a2d-a5d9-b85ca4f0377a None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:52.473571 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d9011-f97d-4a2d-a5d9-b85ca4f0377a None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:52.473981 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d9011-f97d-4a2d-a5d9-b85ca4f0377a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:52.474616 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193d9011-f97d-4a2d-a5d9-b85ca4f0377a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:52.476316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193d9011-f97d-4a2d-a5d9-b85ca4f0377a None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:52.488798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-193d9011-f97d-4a2d-a5d9-b85ca4f0377a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 14:07:52.488798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:52.488798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 14:07:52.488798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 14:07:52.488798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 14:07:52.488798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 14:07:52.488798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 14:07:52.488798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:07:52.488798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 14:07:52.488798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:52.488798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 14:07:52.488798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:52.488798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:52.488798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:52.488798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:52.488798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:52.490027 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:52.490027 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:52.490027 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:52.490027 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:52.490027 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:52.490027 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:52.490027 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:52.490027 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 14:07:52.490027 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 14:07:52.490027 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 14:07:52.490027 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 14:07:52.490027 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:52.490027 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:52.490027 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:52.490027 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:52.490027 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:52.491006 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:52.491006 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:52.491006 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:52.491006 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:52.491006 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:52.491006 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:52.491006 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:52.491006 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:52.491006 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:52.491006 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:52.491006 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:52.491006 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 14:07:52.491006 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:07:52.491006 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 14:07:52.491006 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:52.491006 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 446/866] 10.222.0.26 () {56 vars in 1132 bytes} [Fri Aug 5 14:07:52 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:07:52.496253 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d9011-f97d-4a2d-a5d9-b85ca4f0377a None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:52.589875 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d459b-4e1d-4987-8482-2c26b319b204 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:52.623184 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d9011-f97d-4a2d-a5d9-b85ca4f0377a None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:52.634735 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d04d7-a1e4-4523-89a7-5291bd0e6cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:52.635379 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d04d7-a1e4-4523-89a7-5291bd0e6cbc None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:52.636041 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d04d7-a1e4-4523-89a7-5291bd0e6cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:52.636435 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d04d7-a1e4-4523-89a7-5291bd0e6cbc None admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:52.637364 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76d04d7-a1e4-4523-89a7-5291bd0e6cbc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:52.637771 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76d04d7-a1e4-4523-89a7-5291bd0e6cbc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:52.639523 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76d04d7-a1e4-4523-89a7-5291bd0e6cbc None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:52.655376 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 421/867] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Aug 5 14:07:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:52.660909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d04d7-a1e4-4523-89a7-5291bd0e6cbc None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:52.668947 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c93b6d-bd2f-46d0-a46b-b416293caefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:52.669820 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c93b6d-bd2f-46d0-a46b-b416293caefd None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:52.670242 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c93b6d-bd2f-46d0-a46b-b416293caefd None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:52.670667 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c93b6d-bd2f-46d0-a46b-b416293caefd None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:52.671532 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c93b6d-bd2f-46d0-a46b-b416293caefd None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:52.671991 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c93b6d-bd2f-46d0-a46b-b416293caefd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:52.673578 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c93b6d-bd2f-46d0-a46b-b416293caefd None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:52.679415 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 447/868] 10.222.0.26 () {56 vars in 1126 bytes} [Fri Aug 5 14:07:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:52.686821 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c93b6d-bd2f-46d0-a46b-b416293caefd None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:52.690139 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:52.732780 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:52.766276 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d04d7-a1e4-4523-89a7-5291bd0e6cbc None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:52.790525 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c93b6d-bd2f-46d0-a46b-b416293caefd None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:52.803685 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abfbfb5-edb2-4e6b-b8b0-a34a52607ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:52.804303 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abfbfb5-edb2-4e6b-b8b0-a34a52607ddc None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:52.804628 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abfbfb5-edb2-4e6b-b8b0-a34a52607ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:52.804885 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abfbfb5-edb2-4e6b-b8b0-a34a52607ddc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:52.805674 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abfbfb5-edb2-4e6b-b8b0-a34a52607ddc None admin] RBAC: Adding query filter params (role.id=299c794c7acb4a5ca36288c3191aad10, scope.project.id=319973e011c54b4ca1f2198c8a5e135b, user.id=dddd7eea67f34f0cbb5f6a1cd85f1fc5) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:52.805981 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abfbfb5-edb2-4e6b-b8b0-a34a52607ddc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:52.807416 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abfbfb5-edb2-4e6b-b8b0-a34a52607ddc None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:52.813950 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 422/869] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:07:52 2022] GET /identity/v3/role_assignments?user.id=dddd7eea67f34f0cbb5f6a1cd85f1fc5&scope.project.id=319973e011c54b4ca1f2198c8a5e135b&role.id=299c794c7acb4a5ca36288c3191aad10 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:52.830267 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4094b4-ab53-4190-a0b9-baead9e941e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:52.831867 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4094b4-ab53-4190-a0b9-baead9e941e1 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:52.832332 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4094b4-ab53-4190-a0b9-baead9e941e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:52.832774 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4094b4-ab53-4190-a0b9-baead9e941e1 None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:52.833635 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4094b4-ab53-4190-a0b9-baead9e941e1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:52.834121 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4094b4-ab53-4190-a0b9-baead9e941e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:52.835679 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4094b4-ab53-4190-a0b9-baead9e941e1 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:52.842026 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 448/870] 10.222.0.26 () {56 vars in 1126 bytes} [Fri Aug 5 14:07:52 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:53.405580 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac81beb-8d3d-4704-9d02-0c9767a2a7e9 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:53.406103 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac81beb-8d3d-4704-9d02-0c9767a2a7e9 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:53.406103 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac81beb-8d3d-4704-9d02-0c9767a2a7e9 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:53.407736 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 423/871] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:53.413588 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc13a955-b549-4ad0-b174-0c3cf6dae354 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:53.414015 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc13a955-b549-4ad0-b174-0c3cf6dae354 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:53.414192 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc13a955-b549-4ad0-b174-0c3cf6dae354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:53.423542 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653b0c1-1c21-4487-9f3e-8cc0f8101e98 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:53.423826 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653b0c1-1c21-4487-9f3e-8cc0f8101e98 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:53.424105 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653b0c1-1c21-4487-9f3e-8cc0f8101e98 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:53.425812 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 424/872] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:07:53.431723 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae507e27-b88f-4bab-a059-245a82c0f1c2 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:53.432003 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae507e27-b88f-4bab-a059-245a82c0f1c2 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:53.432273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae507e27-b88f-4bab-a059-245a82c0f1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:53.433552 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-bc13a955-b549-4ad0-b174-0c3cf6dae354 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:53.434753 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-bc13a955-b549-4ad0-b174-0c3cf6dae354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:53.450299 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-ae507e27-b88f-4bab-a059-245a82c0f1c2 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:53.451474 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ae507e27-b88f-4bab-a059-245a82c0f1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:53.528431 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 449/873] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:53 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:07:53.540647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 425/874] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:53 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 14:07:53.545706 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc13a955-b549-4ad0-b174-0c3cf6dae354 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:53.566284 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae507e27-b88f-4bab-a059-245a82c0f1c2 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:53.590817 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:53.594162 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:53.664073 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc13a955-b549-4ad0-b174-0c3cf6dae354 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:53.682295 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae507e27-b88f-4bab-a059-245a82c0f1c2 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:53.707925 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690513a9-4b9c-48d7-809c-709a724cfc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:53.708794 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690513a9-4b9c-48d7-809c-709a724cfc69 None admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:53.709372 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690513a9-4b9c-48d7-809c-709a724cfc69 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:53.709788 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690513a9-4b9c-48d7-809c-709a724cfc69 None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:53.711166 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690513a9-4b9c-48d7-809c-709a724cfc69 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:53.712864 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690513a9-4b9c-48d7-809c-709a724cfc69 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:53.721081 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49343e9-2102-4b03-945f-ed944fcb3165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:53.721910 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49343e9-2102-4b03-945f-ed944fcb3165 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:53.722272 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49343e9-2102-4b03-945f-ed944fcb3165 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:53.722650 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49343e9-2102-4b03-945f-ed944fcb3165 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:53.733071 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49343e9-2102-4b03-945f-ed944fcb3165 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:07:53.738512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-d49343e9-2102-4b03-945f-ed944fcb3165 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 14:07:53.738512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:53.738512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:53.738512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:53.738512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:53.738512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:53.738512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:53.738512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:53.738512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:53.738512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:53.738512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:53.738512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:53.738512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:07:53.738512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:07:53.738512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:07:53.738512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:07:53.739435 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:07:53.739435 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:07:53.739435 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:07:53.739435 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:07:53.739435 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:53.739435 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:53.739435 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:53.739435 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:53.739435 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:53.739435 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:53.739435 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:53.739435 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:53.739435 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:53.739435 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:53.739435 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:53.739435 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:53.740279 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:53.740279 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:53.740279 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:53.740279 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:53.740279 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:07:53.740279 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:07:53.740279 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:07:53.740279 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:07:53.740279 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:07:53.740279 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:07:53.740279 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 14:07:53.740279 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:53.741686 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 426/875] 10.222.0.26 () {56 vars in 1120 bytes} [Fri Aug 5 14:07:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:53.745776 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49343e9-2102-4b03-945f-ed944fcb3165 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:53.784486 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 450/876] 10.222.0.26 () {60 vars in 1141 bytes} [Fri Aug 5 14:07:53 2022] POST /identity/v3/services => generated 251 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:07:53.858283 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:53.880928 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49343e9-2102-4b03-945f-ed944fcb3165 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:53.929831 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868e11f-c45a-4c57-a5ba-445a2a9f2489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:53.930386 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868e11f-c45a-4c57-a5ba-445a2a9f2489 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:53.930662 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868e11f-c45a-4c57-a5ba-445a2a9f2489 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:53.930932 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868e11f-c45a-4c57-a5ba-445a2a9f2489 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:53.931661 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a868e11f-c45a-4c57-a5ba-445a2a9f2489 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:53.931964 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a868e11f-c45a-4c57-a5ba-445a2a9f2489 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:53.933600 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a868e11f-c45a-4c57-a5ba-445a2a9f2489 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:53.948108 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 427/877] 10.222.0.26 () {56 vars in 1115 bytes} [Fri Aug 5 14:07:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:53.953309 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690513a9-4b9c-48d7-809c-709a724cfc69 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:54.062232 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:54.067157 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690513a9-4b9c-48d7-809c-709a724cfc69 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:54.102473 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0cb3ac-cb68-41d9-a477-da362b56d8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:54.103112 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0cb3ac-cb68-41d9-a477-da362b56d8ba None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:54.103385 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0cb3ac-cb68-41d9-a477-da362b56d8ba None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:54.103652 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0cb3ac-cb68-41d9-a477-da362b56d8ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:54.115513 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0cb3ac-cb68-41d9-a477-da362b56d8ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:54.117116 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0cb3ac-cb68-41d9-a477-da362b56d8ba None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:54.131895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-bd0cb3ac-cb68-41d9-a477-da362b56d8ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:54.131895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:54.131895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:54.131895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:54.131895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:54.131895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:54.131895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:54.131895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:54.131895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:54.131895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:54.131895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:54.131895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:54.131895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:54.131895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:54.131895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:54.132866 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:54.132866 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:54.132866 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:54.132866 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:54.132866 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:54.132866 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:54.132866 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:54.132866 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:54.132866 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:54.132866 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:54.132866 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:54.132866 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:54.132866 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:54.132866 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:54.132866 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:54.132866 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:54.132866 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:54.133782 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:54.133782 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:54.133782 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:54.133782 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:54.133782 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 451/878] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:54.140878 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868e11f-c45a-4c57-a5ba-445a2a9f2489 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:54.251069 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868e11f-c45a-4c57-a5ba-445a2a9f2489 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:54.285426 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807a653-959d-4aaf-87cf-5bc5451aff6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:54.286033 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807a653-959d-4aaf-87cf-5bc5451aff6d None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:54.286372 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807a653-959d-4aaf-87cf-5bc5451aff6d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:54.286645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807a653-959d-4aaf-87cf-5bc5451aff6d None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:54.287326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2807a653-959d-4aaf-87cf-5bc5451aff6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:54.287673 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2807a653-959d-4aaf-87cf-5bc5451aff6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:54.289055 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2807a653-959d-4aaf-87cf-5bc5451aff6d None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:54.296168 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:54.305654 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 428/879] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:54.311462 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0cb3ac-cb68-41d9-a477-da362b56d8ba None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:54.420133 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0cb3ac-cb68-41d9-a477-da362b56d8ba None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:54.460818 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6dbcfb-3dea-43ef-b23b-7a39a9c49845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:54.461504 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6dbcfb-3dea-43ef-b23b-7a39a9c49845 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:54.461780 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6dbcfb-3dea-43ef-b23b-7a39a9c49845 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:54.462046 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6dbcfb-3dea-43ef-b23b-7a39a9c49845 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:54.470100 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6dbcfb-3dea-43ef-b23b-7a39a9c49845 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:54.472074 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6dbcfb-3dea-43ef-b23b-7a39a9c49845 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:54.483994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-3b6dbcfb-3dea-43ef-b23b-7a39a9c49845 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 14:07:54.483994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:54.483994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:54.483994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:54.483994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:54.483994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:54.483994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:54.483994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:54.483994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:54.483994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:54.483994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:54.483994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:54.483994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 14:07:54.483994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 14:07:54.483994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 14:07:54.483994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 14:07:54.484969 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:54.484969 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:54.484969 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:07:54.484969 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:54.484969 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:07:54.484969 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:54.484969 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:54.484969 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:54.484969 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:54.484969 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:54.484969 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:54.484969 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:54.484969 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:54.484969 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:54.484969 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:54.484969 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:54.484969 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:54.484969 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:54.485906 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:54.485906 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:54.485906 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 14:07:54.485906 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 14:07:54.485906 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 14:07:54.485906 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 14:07:54.485906 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 14:07:54.485906 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 14:07:54.485906 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 14:07:54.485906 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:54.485906 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 452/880] 10.222.0.26 () {56 vars in 1117 bytes} [Fri Aug 5 14:07:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:54.493458 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807a653-959d-4aaf-87cf-5bc5451aff6d None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:54.606269 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807a653-959d-4aaf-87cf-5bc5451aff6d None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:54.642186 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6aa0eb-7c70-4138-a83f-1205c32e9bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:54.642796 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6aa0eb-7c70-4138-a83f-1205c32e9bde None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:54.643134 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6aa0eb-7c70-4138-a83f-1205c32e9bde None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:54.643935 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6aa0eb-7c70-4138-a83f-1205c32e9bde None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:54.644721 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6aa0eb-7c70-4138-a83f-1205c32e9bde None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:54.645022 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6aa0eb-7c70-4138-a83f-1205c32e9bde None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:54.646432 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6aa0eb-7c70-4138-a83f-1205c32e9bde None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:54.664596 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 429/881] 10.222.0.26 () {56 vars in 1149 bytes} [Fri Aug 5 14:07:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:54.673494 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6dbcfb-3dea-43ef-b23b-7a39a9c49845 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:54.687034 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:54.781808 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6dbcfb-3dea-43ef-b23b-7a39a9c49845 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:54.818701 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f7907-a4ba-4b46-8509-692df17d21e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:54.819318 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f7907-a4ba-4b46-8509-692df17d21e5 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:54.819596 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f7907-a4ba-4b46-8509-692df17d21e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:54.819866 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f7907-a4ba-4b46-8509-692df17d21e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:54.902001 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26f7907-a4ba-4b46-8509-692df17d21e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:54.903460 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26f7907-a4ba-4b46-8509-692df17d21e5 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:54.915761 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:54.928473 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd23893-7d1b-44cd-8dff-54b5cd0ce303 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:54.928768 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd23893-7d1b-44cd-8dff-54b5cd0ce303 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:54.929020 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd23893-7d1b-44cd-8dff-54b5cd0ce303 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:54.931581 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 430/882] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:54.933911 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85887264-9e46-4695-97a0-c5c569d49154 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:54.934712 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85887264-9e46-4695-97a0-c5c569d49154 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:54.935001 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85887264-9e46-4695-97a0-c5c569d49154 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:54.936322 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 431/883] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:54.937275 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f639c5-a6ac-4c3d-a36b-3a2d1e7e3f7c None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:54.937551 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f639c5-a6ac-4c3d-a36b-3a2d1e7e3f7c None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:54.937850 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f639c5-a6ac-4c3d-a36b-3a2d1e7e3f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:54.960682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-19f639c5-a6ac-4c3d-a36b-3a2d1e7e3f7c None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:54.961875 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-19f639c5-a6ac-4c3d-a36b-3a2d1e7e3f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:55.018578 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 453/884] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:07:54 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 14:07:55.019923 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b848a54-6710-4527-a330-39e0b43c7b35 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:55.020210 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b848a54-6710-4527-a330-39e0b43c7b35 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:55.020517 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b848a54-6710-4527-a330-39e0b43c7b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:55.024557 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:55.039222 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-8b848a54-6710-4527-a330-39e0b43c7b35 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:55.040335 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-8b848a54-6710-4527-a330-39e0b43c7b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:55.042651 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 432/885] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:54 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:07:55.043315 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f639c5-a6ac-4c3d-a36b-3a2d1e7e3f7c None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:55.113641 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 454/886] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:55 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:07:55.114474 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b848a54-6710-4527-a330-39e0b43c7b35 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:55.169863 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f639c5-a6ac-4c3d-a36b-3a2d1e7e3f7c None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:55.171564 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:55.206745 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b284adb-7797-4789-bdc9-9fd6488bf27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:55.207738 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b284adb-7797-4789-bdc9-9fd6488bf27d None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:55.208231 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b284adb-7797-4789-bdc9-9fd6488bf27d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:55.208707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b284adb-7797-4789-bdc9-9fd6488bf27d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:55.219470 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b284adb-7797-4789-bdc9-9fd6488bf27d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:55.221778 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b284adb-7797-4789-bdc9-9fd6488bf27d None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:55.232201 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b848a54-6710-4527-a330-39e0b43c7b35 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:55.238138 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-7b284adb-7797-4789-bdc9-9fd6488bf27d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:55.238138 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:55.238138 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:55.238138 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:55.238138 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:55.238138 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:55.238138 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:55.238138 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:55.238138 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:55.238138 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:55.238138 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:55.238138 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:55.238138 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:55.238138 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:55.238138 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:55.239026 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:55.239026 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:55.239026 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:55.239026 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:55.239026 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:55.239026 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:55.239026 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:55.239026 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:55.239026 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:55.239026 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:55.239026 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:55.239026 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:55.239026 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:55.239026 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:55.239026 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:55.239026 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:55.239026 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:55.239938 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:55.239938 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:55.239938 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:55.239938 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:55.241911 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 433/887] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:55.242828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b284adb-7797-4789-bdc9-9fd6488bf27d None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:55.273150 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fffe19-b492-4577-87e3-9662389aaa01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:55.273764 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fffe19-b492-4577-87e3-9662389aaa01 None admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:55.274043 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fffe19-b492-4577-87e3-9662389aaa01 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:55.274313 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fffe19-b492-4577-87e3-9662389aaa01 None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:55.275002 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fffe19-b492-4577-87e3-9662389aaa01 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:55.276349 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fffe19-b492-4577-87e3-9662389aaa01 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:55.284250 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:55.340743 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 455/888] 10.222.0.26 () {60 vars in 1141 bytes} [Fri Aug 5 14:07:55 2022] POST /identity/v3/services => generated 269 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:07:55.341547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fffe19-b492-4577-87e3-9662389aaa01 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:55.342100 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b284adb-7797-4789-bdc9-9fd6488bf27d None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:55.395891 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e785f6-2b06-4c15-b695-3d6d09619a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:55.396900 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e785f6-2b06-4c15-b695-3d6d09619a54 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:55.397031 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e785f6-2b06-4c15-b695-3d6d09619a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:55.397373 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e785f6-2b06-4c15-b695-3d6d09619a54 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:55.398196 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e785f6-2b06-4c15-b695-3d6d09619a54 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:55.399783 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e785f6-2b06-4c15-b695-3d6d09619a54 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:55.411039 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-d3e785f6-2b06-4c15-b695-3d6d09619a54 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 14:07:55.411039 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:55.411039 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 14:07:55.411039 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 14:07:55.411039 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 14:07:55.411039 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 14:07:55.411039 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 14:07:55.411039 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:07:55.411039 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 14:07:55.411039 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:55.411039 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 14:07:55.411039 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:55.411039 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:55.411039 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:55.411039 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:55.411039 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:55.411039 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:55.412127 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:55.412127 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:55.412127 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:55.412127 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:55.412127 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:55.412127 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:55.412127 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 14:07:55.412127 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 14:07:55.412127 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 14:07:55.412127 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 14:07:55.412127 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:55.412127 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:55.412127 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:55.412127 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:55.412127 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:55.412127 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:55.413003 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:55.413003 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:55.413003 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:55.413003 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:55.413003 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:55.413003 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:55.413003 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:55.413003 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:55.413003 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:55.413003 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:55.413003 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 14:07:55.413003 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:07:55.413003 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 14:07:55.413003 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:55.413003 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 434/889] 10.222.0.26 () {56 vars in 1126 bytes} [Fri Aug 5 14:07:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:55.418069 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e785f6-2b06-4c15-b695-3d6d09619a54 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:55.476791 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:55.481596 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fffe19-b492-4577-87e3-9662389aaa01 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:55.526198 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1a02f8-b1ae-45a5-9520-cc322d94a3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:55.526334 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a02f8-b1ae-45a5-9520-cc322d94a3c5 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:55.526624 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a02f8-b1ae-45a5-9520-cc322d94a3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:55.526901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a02f8-b1ae-45a5-9520-cc322d94a3c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:55.528066 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1a02f8-b1ae-45a5-9520-cc322d94a3c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:55.528243 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1a02f8-b1ae-45a5-9520-cc322d94a3c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:55.530202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1a02f8-b1ae-45a5-9520-cc322d94a3c5 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:55.534541 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:55.538396 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e785f6-2b06-4c15-b695-3d6d09619a54 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:55.546344 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 456/890] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:55.554402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1a02f8-b1ae-45a5-9520-cc322d94a3c5 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:55.586100 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206cc098-e680-4339-925c-d7003f7b3ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:55.587083 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206cc098-e680-4339-925c-d7003f7b3ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:55.588260 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206cc098-e680-4339-925c-d7003f7b3ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:55.588732 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206cc098-e680-4339-925c-d7003f7b3ea0 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:55.590244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206cc098-e680-4339-925c-d7003f7b3ea0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:55.591039 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206cc098-e680-4339-925c-d7003f7b3ea0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:55.593745 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206cc098-e680-4339-925c-d7003f7b3ea0 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:55.603322 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 435/891] 10.222.0.26 () {56 vars in 1122 bytes} [Fri Aug 5 14:07:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:55.610067 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206cc098-e680-4339-925c-d7003f7b3ea0 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:55.676381 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1a02f8-b1ae-45a5-9520-cc322d94a3c5 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:55.716935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb89dc9-8034-47d8-99db-823262170f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:55.717907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb89dc9-8034-47d8-99db-823262170f82 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:55.718362 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb89dc9-8034-47d8-99db-823262170f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:55.718815 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb89dc9-8034-47d8-99db-823262170f82 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:55.726509 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb89dc9-8034-47d8-99db-823262170f82 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:55.727937 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb89dc9-8034-47d8-99db-823262170f82 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:55.733464 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206cc098-e680-4339-925c-d7003f7b3ea0 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:55.742902 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-feb89dc9-8034-47d8-99db-823262170f82 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:55.742902 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:55.742902 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:55.742902 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:55.742902 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:55.742902 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:55.742902 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:55.742902 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:55.742902 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:55.742902 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:55.742902 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:55.742902 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:55.742902 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 14:07:55.742902 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 14:07:55.742902 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 14:07:55.743821 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 14:07:55.743821 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:55.743821 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:55.743821 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:55.743821 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:55.743821 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:55.743821 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:55.743821 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:55.743821 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:55.743821 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:55.743821 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:55.743821 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:55.743821 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:55.743821 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:55.743821 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:55.743821 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:55.743821 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:55.744751 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 14:07:55.744751 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 14:07:55.744751 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:07:55.744751 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:07:55.744751 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:07:55.744751 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:07:55.744751 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:07:55.744751 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:55.751860 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 457/892] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:07:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:55.754112 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb89dc9-8034-47d8-99db-823262170f82 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:55.775604 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d865edf-e736-4820-b3a5-8bf3b9589d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:55.776230 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d865edf-e736-4820-b3a5-8bf3b9589d3d None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:55.776601 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d865edf-e736-4820-b3a5-8bf3b9589d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:55.776876 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d865edf-e736-4820-b3a5-8bf3b9589d3d None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:55.777611 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d865edf-e736-4820-b3a5-8bf3b9589d3d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:55.778047 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d865edf-e736-4820-b3a5-8bf3b9589d3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:55.778955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:55.780386 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d865edf-e736-4820-b3a5-8bf3b9589d3d None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:55.786097 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 436/893] 10.222.0.26 () {56 vars in 1122 bytes} [Fri Aug 5 14:07:55 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:55.873019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb89dc9-8034-47d8-99db-823262170f82 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:55.913084 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a237481-3f1d-4b7c-bcac-1dbe936d57ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:55.913764 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a237481-3f1d-4b7c-bcac-1dbe936d57ed None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:55.914044 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a237481-3f1d-4b7c-bcac-1dbe936d57ed None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:55.914314 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a237481-3f1d-4b7c-bcac-1dbe936d57ed None admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:55.915075 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a237481-3f1d-4b7c-bcac-1dbe936d57ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:55.915379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a237481-3f1d-4b7c-bcac-1dbe936d57ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:55.916859 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a237481-3f1d-4b7c-bcac-1dbe936d57ed None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:55.936058 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 458/894] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Aug 5 14:07:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:55.937704 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d865edf-e736-4820-b3a5-8bf3b9589d3d None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:55.976066 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:55.998408 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb3bee-86a1-47ae-ace8-e7f53923001e None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:55.998712 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb3bee-86a1-47ae-ace8-e7f53923001e None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:55.998964 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb3bee-86a1-47ae-ace8-e7f53923001e None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:56.000586 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 459/895] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:07:56.006091 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192799af-0838-4123-9029-1f611e3d3302 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:56.006373 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192799af-0838-4123-9029-1f611e3d3302 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:56.006651 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192799af-0838-4123-9029-1f611e3d3302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:56.025877 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-192799af-0838-4123-9029-1f611e3d3302 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:56.027071 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-192799af-0838-4123-9029-1f611e3d3302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:56.063066 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d865edf-e736-4820-b3a5-8bf3b9589d3d None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:56.104556 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 460/896] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:56 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 14:07:56.106884 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046a4601-8de9-4b50-a3a1-dc7f7e402173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:56.107515 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a4601-8de9-4b50-a3a1-dc7f7e402173 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:56.107931 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a4601-8de9-4b50-a3a1-dc7f7e402173 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:56.108270 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a4601-8de9-4b50-a3a1-dc7f7e402173 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:56.109176 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046a4601-8de9-4b50-a3a1-dc7f7e402173 None admin] RBAC: Adding query filter params (role.id=5c0fa9dd58a349cba21555779c761f0b, scope.project.id=319973e011c54b4ca1f2198c8a5e135b, user.id=17e3b509a2de4159bf234594573f5e18) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:56.109551 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046a4601-8de9-4b50-a3a1-dc7f7e402173 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:56.111051 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046a4601-8de9-4b50-a3a1-dc7f7e402173 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:56.117421 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 437/897] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:07:55 2022] GET /identity/v3/role_assignments?user.id=17e3b509a2de4159bf234594573f5e18&scope.project.id=319973e011c54b4ca1f2198c8a5e135b&role.id=5c0fa9dd58a349cba21555779c761f0b => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:56.124722 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192799af-0838-4123-9029-1f611e3d3302 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:56.138862 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:56.241087 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192799af-0838-4123-9029-1f611e3d3302 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:56.281053 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c1978b-6f77-4781-91dc-7cc5a0a998af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:56.281700 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c1978b-6f77-4781-91dc-7cc5a0a998af None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:56.281977 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c1978b-6f77-4781-91dc-7cc5a0a998af None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:56.282275 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c1978b-6f77-4781-91dc-7cc5a0a998af None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:56.282883 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c1978b-6f77-4781-91dc-7cc5a0a998af None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:56.285103 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c1978b-6f77-4781-91dc-7cc5a0a998af None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:56.294225 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-a2c1978b-6f77-4781-91dc-7cc5a0a998af None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 14:07:56.294225 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:56.294225 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 14:07:56.294225 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 14:07:56.294225 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 14:07:56.294225 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 14:07:56.294225 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 14:07:56.294225 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:07:56.294225 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 14:07:56.294225 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:56.294225 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 14:07:56.294225 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:56.294225 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:56.294225 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:56.294225 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:56.294225 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:56.294225 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:56.295202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:56.295202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:56.295202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:56.295202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:56.295202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:56.295202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:56.295202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 14:07:56.295202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 14:07:56.295202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 14:07:56.295202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 14:07:56.295202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:56.295202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:56.295202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:56.295202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:56.295202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:56.295202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:56.296649 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:56.296649 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:56.296649 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:56.296649 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:56.296649 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:56.296649 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:56.296649 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:56.296649 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:56.296649 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:56.296649 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:56.296649 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 14:07:56.296649 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:07:56.296649 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 14:07:56.296649 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:56.296649 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 461/898] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Aug 5 14:07:56 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:56.301360 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046a4601-8de9-4b50-a3a1-dc7f7e402173 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:56.397894 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:56.407415 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046a4601-8de9-4b50-a3a1-dc7f7e402173 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:56.443662 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711564bd-ea7a-487a-a4b8-901777bb78b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:56.444282 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711564bd-ea7a-487a-a4b8-901777bb78b9 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:56.444595 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711564bd-ea7a-487a-a4b8-901777bb78b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:56.444869 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711564bd-ea7a-487a-a4b8-901777bb78b9 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:56.445547 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711564bd-ea7a-487a-a4b8-901777bb78b9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:56.445849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711564bd-ea7a-487a-a4b8-901777bb78b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:56.447276 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711564bd-ea7a-487a-a4b8-901777bb78b9 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:56.452479 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 438/899] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Aug 5 14:07:56 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:56.461497 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c1978b-6f77-4781-91dc-7cc5a0a998af None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:56.560395 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:56.586101 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c1978b-6f77-4781-91dc-7cc5a0a998af None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:56.621759 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47267d8-1437-44bb-803b-444c962ba587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:56.622377 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47267d8-1437-44bb-803b-444c962ba587 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:56.622653 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47267d8-1437-44bb-803b-444c962ba587 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:56.622954 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47267d8-1437-44bb-803b-444c962ba587 None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:56.623632 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47267d8-1437-44bb-803b-444c962ba587 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:56.623934 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47267d8-1437-44bb-803b-444c962ba587 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:56.625354 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47267d8-1437-44bb-803b-444c962ba587 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:56.631508 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 462/900] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Aug 5 14:07:56 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:56.637622 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711564bd-ea7a-487a-a4b8-901777bb78b9 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:56.743218 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711564bd-ea7a-487a-a4b8-901777bb78b9 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:56.778698 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d707c-387e-40f0-9a21-0c39ad37cadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:56.779322 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d707c-387e-40f0-9a21-0c39ad37cadd None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:56.779603 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d707c-387e-40f0-9a21-0c39ad37cadd None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:56.779878 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d707c-387e-40f0-9a21-0c39ad37cadd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:56.780657 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021d707c-387e-40f0-9a21-0c39ad37cadd None admin] RBAC: Adding query filter params (interface=public, service_id=ec52651523344f62af1475a4deca979b, region_id=RegionOne) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:56.780967 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021d707c-387e-40f0-9a21-0c39ad37cadd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:56.782406 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021d707c-387e-40f0-9a21-0c39ad37cadd None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:56.787870 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 439/901] 10.222.0.26 () {56 vars in 1251 bytes} [Fri Aug 5 14:07:56 2022] GET /identity/v3/endpoints?service_id=ec52651523344f62af1475a4deca979b&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:56.793332 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47267d8-1437-44bb-803b-444c962ba587 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:56.897792 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47267d8-1437-44bb-803b-444c962ba587 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:56.936816 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065a9c3e-8cea-40cd-8d68-99bc007ed1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:56.937433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a9c3e-8cea-40cd-8d68-99bc007ed1bf None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:56.937713 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a9c3e-8cea-40cd-8d68-99bc007ed1bf None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:56.937985 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a9c3e-8cea-40cd-8d68-99bc007ed1bf None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:56.938637 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065a9c3e-8cea-40cd-8d68-99bc007ed1bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:56.940212 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065a9c3e-8cea-40cd-8d68-99bc007ed1bf None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:56.946307 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 463/902] 10.222.0.26 () {56 vars in 1087 bytes} [Fri Aug 5 14:07:56 2022] GET /identity/v3/services => generated 1268 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:07:57.581509 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1befc-5ff6-4fee-aef3-232340b1cee2 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:57.581977 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1befc-5ff6-4fee-aef3-232340b1cee2 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:57.582086 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1befc-5ff6-4fee-aef3-232340b1cee2 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:57.583756 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 440/903] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:57.589110 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e3502-c514-4705-98d7-dc2201bc5f85 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:57.589570 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e3502-c514-4705-98d7-dc2201bc5f85 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:57.589650 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e3502-c514-4705-98d7-dc2201bc5f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:57.596078 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:57.607992 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-147e3502-c514-4705-98d7-dc2201bc5f85 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:57.610364 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-147e3502-c514-4705-98d7-dc2201bc5f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:57.682694 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 464/904] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:57 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:07:57.706019 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1befc-5ff6-4fee-aef3-232340b1cee2 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:57.725994 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:57.814329 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1befc-5ff6-4fee-aef3-232340b1cee2 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:57.850487 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfefd420-b24b-4a0b-bde3-b7ec0c9dabb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:57.851267 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefd420-b24b-4a0b-bde3-b7ec0c9dabb2 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:57.851447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefd420-b24b-4a0b-bde3-b7ec0c9dabb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:57.851713 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefd420-b24b-4a0b-bde3-b7ec0c9dabb2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:57.852341 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfefd420-b24b-4a0b-bde3-b7ec0c9dabb2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:57.853782 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfefd420-b24b-4a0b-bde3-b7ec0c9dabb2 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:57.863519 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-bfefd420-b24b-4a0b-bde3-b7ec0c9dabb2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 14:07:57.863519 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:57.863519 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 14:07:57.863519 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 14:07:57.863519 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 14:07:57.863519 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 14:07:57.863519 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 14:07:57.863519 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:07:57.863519 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 14:07:57.863519 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:57.863519 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 14:07:57.863519 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:57.863519 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:57.863519 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:57.863519 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:57.863519 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:57.870193 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:57.870193 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:57.870193 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:57.870193 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:57.870193 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:57.870193 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:57.870193 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:57.870193 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 14:07:57.870193 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 14:07:57.870193 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 14:07:57.870193 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 14:07:57.870193 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:57.870193 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:57.870193 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:57.870193 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:57.870193 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:57.871082 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:57.871082 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:57.871082 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:57.871082 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:57.871082 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:57.871082 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:57.871082 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:57.871082 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:57.871082 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:57.871082 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:57.871082 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:57.871082 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 14:07:57.871082 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:07:57.871082 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 14:07:57.871082 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:57.871082 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 441/905] 10.222.0.26 () {56 vars in 1132 bytes} [Fri Aug 5 14:07:57 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:57.871966 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e3502-c514-4705-98d7-dc2201bc5f85 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:57.973194 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46470257-e7dd-4d93-a7e3-608e20fd4840 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:57.973532 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46470257-e7dd-4d93-a7e3-608e20fd4840 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:57.973752 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46470257-e7dd-4d93-a7e3-608e20fd4840 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:57.975441 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 442/906] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:07:57.977813 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e3502-c514-4705-98d7-dc2201bc5f85 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:57.980798 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f647d8-ab3a-4e46-a6cc-ac520c994233 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:57.981089 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f647d8-ab3a-4e46-a6cc-ac520c994233 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:57.981428 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f647d8-ab3a-4e46-a6cc-ac520c994233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:58.004710 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-c1f647d8-ab3a-4e46-a6cc-ac520c994233 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:58.005965 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c1f647d8-ab3a-4e46-a6cc-ac520c994233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:58.014203 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:58.017022 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf575310-a0f4-48ec-a5c9-b9e19aad364f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:58.017650 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf575310-a0f4-48ec-a5c9-b9e19aad364f None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:58.018562 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf575310-a0f4-48ec-a5c9-b9e19aad364f None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:58.018843 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf575310-a0f4-48ec-a5c9-b9e19aad364f None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:58.019529 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf575310-a0f4-48ec-a5c9-b9e19aad364f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:58.019835 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf575310-a0f4-48ec-a5c9-b9e19aad364f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:58.021327 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf575310-a0f4-48ec-a5c9-b9e19aad364f None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:58.026647 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 465/907] 10.222.0.26 () {56 vars in 1126 bytes} [Fri Aug 5 14:07:57 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:58.033061 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf575310-a0f4-48ec-a5c9-b9e19aad364f None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:58.094496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 443/908] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:57 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 14:07:58.118061 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f647d8-ab3a-4e46-a6cc-ac520c994233 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:58.137398 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:58.161177 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf575310-a0f4-48ec-a5c9-b9e19aad364f None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:58.197034 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f57b0-1129-4385-a34d-6322966d7aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:58.197647 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f57b0-1129-4385-a34d-6322966d7aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:58.197926 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f57b0-1129-4385-a34d-6322966d7aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:58.198198 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f57b0-1129-4385-a34d-6322966d7aa7 None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:58.198875 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65f57b0-1129-4385-a34d-6322966d7aa7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:58.199173 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65f57b0-1129-4385-a34d-6322966d7aa7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:58.200726 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65f57b0-1129-4385-a34d-6322966d7aa7 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:58.207089 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 466/909] 10.222.0.26 () {56 vars in 1126 bytes} [Fri Aug 5 14:07:58 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:58.208287 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bafc6d-aa95-464e-b161-3c499b289348 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:58.208593 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bafc6d-aa95-464e-b161-3c499b289348 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:58.208864 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bafc6d-aa95-464e-b161-3c499b289348 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:58.212405 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 467/910] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:07:58.212405 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bafc6d-aa95-464e-b161-3c499b289348 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:58.233179 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f647d8-ab3a-4e46-a6cc-ac520c994233 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:58.272036 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f0aecc-9f54-4a51-9318-fee0f271aa92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:58.272831 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f0aecc-9f54-4a51-9318-fee0f271aa92 None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:58.273108 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f0aecc-9f54-4a51-9318-fee0f271aa92 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:58.273600 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:58.273713 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f0aecc-9f54-4a51-9318-fee0f271aa92 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:58.274499 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f0aecc-9f54-4a51-9318-fee0f271aa92 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:58.276165 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f0aecc-9f54-4a51-9318-fee0f271aa92 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:58.315618 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bafc6d-aa95-464e-b161-3c499b289348 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:58.348498 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 444/911] 10.222.0.26 () {60 vars in 1141 bytes} [Fri Aug 5 14:07:58 2022] POST /identity/v3/services => generated 241 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:07:58.354381 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6f2f9-ec6d-4e1f-8686-bf27fd19ec70 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:58.355148 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6f2f9-ec6d-4e1f-8686-bf27fd19ec70 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:58.355698 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6f2f9-ec6d-4e1f-8686-bf27fd19ec70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:58.356065 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88e7fd-092c-4b8a-b37d-966edcfe8e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:58.356795 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88e7fd-092c-4b8a-b37d-966edcfe8e24 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:58.357131 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88e7fd-092c-4b8a-b37d-966edcfe8e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:58.357407 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88e7fd-092c-4b8a-b37d-966edcfe8e24 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:58.358245 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c88e7fd-092c-4b8a-b37d-966edcfe8e24 None admin] RBAC: Adding query filter params (interface=public, service_id=dc1dc8aad7e24492936c9f944530ce6f, region_id=RegionOne) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:58.358608 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c88e7fd-092c-4b8a-b37d-966edcfe8e24 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:58.360001 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c88e7fd-092c-4b8a-b37d-966edcfe8e24 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:58.368012 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 468/912] 10.222.0.26 () {56 vars in 1251 bytes} [Fri Aug 5 14:07:58 2022] GET /identity/v3/endpoints?service_id=dc1dc8aad7e24492936c9f944530ce6f&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:58.373660 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88e7fd-092c-4b8a-b37d-966edcfe8e24 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:58.381555 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-7fc6f2f9-ec6d-4e1f-8686-bf27fd19ec70 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:58.383469 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-7fc6f2f9-ec6d-4e1f-8686-bf27fd19ec70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:58.484443 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 445/913] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:07:58.504652 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc6f2f9-ec6d-4e1f-8686-bf27fd19ec70 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:58.507683 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:58.538083 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88e7fd-092c-4b8a-b37d-966edcfe8e24 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:58.561245 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:58.586177 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a772a-588d-4134-9686-8e8ebbdf2b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:58.586177 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a772a-588d-4134-9686-8e8ebbdf2b16 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:58.586177 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a772a-588d-4134-9686-8e8ebbdf2b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:58.586177 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a772a-588d-4134-9686-8e8ebbdf2b16 None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:58.586649 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a772a-588d-4134-9686-8e8ebbdf2b16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:58.586649 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a772a-588d-4134-9686-8e8ebbdf2b16 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:58.592551 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 469/914] 10.222.0.26 () {56 vars in 1087 bytes} [Fri Aug 5 14:07:58 2022] GET /identity/v3/services => generated 1497 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:07:58.633945 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc6f2f9-ec6d-4e1f-8686-bf27fd19ec70 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:58.676292 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba14022-b06d-4833-90b8-b39256deed4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:58.676989 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba14022-b06d-4833-90b8-b39256deed4f None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:58.677262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba14022-b06d-4833-90b8-b39256deed4f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:58.677532 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba14022-b06d-4833-90b8-b39256deed4f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:58.690528 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba14022-b06d-4833-90b8-b39256deed4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:07:58.697079 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-bba14022-b06d-4833-90b8-b39256deed4f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 14:07:58.697079 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:58.697079 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:58.697079 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:58.697079 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:58.697079 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:58.697079 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:58.697079 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:58.697079 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:58.697079 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:58.697079 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:58.697079 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:58.697079 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:07:58.697079 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:07:58.697079 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:07:58.697079 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:07:58.698063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:07:58.698063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:07:58.698063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:07:58.698063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:07:58.698063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:58.698063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:58.698063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:58.698063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:58.698063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:58.698063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:58.698063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:58.698063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:58.698063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:58.698063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:58.698063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:58.698063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:58.699125 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:58.699125 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:58.699125 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:58.699125 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:58.699125 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:07:58.699125 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:07:58.699125 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:07:58.699125 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:07:58.699125 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:07:58.699125 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:07:58.699125 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 14:07:58.699125 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:58.699125 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 446/915] 10.222.0.26 () {56 vars in 1120 bytes} [Fri Aug 5 14:07:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:58.707668 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a772a-588d-4134-9686-8e8ebbdf2b16 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:58.823040 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a772a-588d-4134-9686-8e8ebbdf2b16 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:58.859869 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3775fb8-9ba7-454a-87fd-dd4a1d11edff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:58.860891 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3775fb8-9ba7-454a-87fd-dd4a1d11edff None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:58.861387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3775fb8-9ba7-454a-87fd-dd4a1d11edff None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:58.862463 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3775fb8-9ba7-454a-87fd-dd4a1d11edff None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:58.863400 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3775fb8-9ba7-454a-87fd-dd4a1d11edff None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:58.863897 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3775fb8-9ba7-454a-87fd-dd4a1d11edff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:58.865465 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3775fb8-9ba7-454a-87fd-dd4a1d11edff None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:58.880560 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 470/916] 10.222.0.26 () {56 vars in 1115 bytes} [Fri Aug 5 14:07:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:58.882479 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba14022-b06d-4833-90b8-b39256deed4f None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:58.950772 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:58.990542 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba14022-b06d-4833-90b8-b39256deed4f None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:59.026916 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2e1a6-5099-4fb7-9ffd-053a5fc5b896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:59.027551 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2e1a6-5099-4fb7-9ffd-053a5fc5b896 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:59.027824 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2e1a6-5099-4fb7-9ffd-053a5fc5b896 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:59.028090 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2e1a6-5099-4fb7-9ffd-053a5fc5b896 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:59.038691 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed2e1a6-5099-4fb7-9ffd-053a5fc5b896 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:59.040129 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed2e1a6-5099-4fb7-9ffd-053a5fc5b896 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:59.052250 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97275293-878d-4c2e-8d8d-5160f7752984 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:59.052573 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97275293-878d-4c2e-8d8d-5160f7752984 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:59.052850 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97275293-878d-4c2e-8d8d-5160f7752984 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:59.054447 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 471/917] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:07:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:07:59.054645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-0ed2e1a6-5099-4fb7-9ffd-053a5fc5b896 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:59.054645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:59.054645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:59.054645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:59.054645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:59.054645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:59.054645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:59.054645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:59.054645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:59.054645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:59.054645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:59.054645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:59.054645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:59.054645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:59.054645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:59.055881 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:59.055881 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:59.055881 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:59.055881 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:59.055881 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:59.055881 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:59.055881 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:59.055881 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:59.055881 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:59.055881 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:59.055881 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:59.055881 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:59.055881 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:59.055881 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:59.055881 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:59.055881 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:59.055881 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:59.056985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:59.056985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:59.056985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:59.056985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:59.056985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 447/918] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:59.059986 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe0157-9862-426c-8bd2-9586aaa9e4f6 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:59.060273 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe0157-9862-426c-8bd2-9586aaa9e4f6 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:59.060587 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe0157-9862-426c-8bd2-9586aaa9e4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:59.066814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2e1a6-5099-4fb7-9ffd-053a5fc5b896 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:59.079772 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-acbe0157-9862-426c-8bd2-9586aaa9e4f6 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:07:59.081578 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-acbe0157-9862-426c-8bd2-9586aaa9e4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:07:59.135967 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:59.175024 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 472/919] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:07:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:07:59.193257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe0157-9862-426c-8bd2-9586aaa9e4f6 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:59.200762 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2e1a6-5099-4fb7-9ffd-053a5fc5b896 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:59.220186 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:59.247589 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6a554-baca-4768-a2b6-a6e2652a9f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:59.248590 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6a554-baca-4768-a2b6-a6e2652a9f3a None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:59.249473 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6a554-baca-4768-a2b6-a6e2652a9f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:59.250001 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6a554-baca-4768-a2b6-a6e2652a9f3a None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:59.251140 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f6a554-baca-4768-a2b6-a6e2652a9f3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:59.251900 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f6a554-baca-4768-a2b6-a6e2652a9f3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:59.254258 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f6a554-baca-4768-a2b6-a6e2652a9f3a None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:59.270969 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 448/920] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:07:59.276264 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6a554-baca-4768-a2b6-a6e2652a9f3a None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:59.317681 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe0157-9862-426c-8bd2-9586aaa9e4f6 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:59.355702 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db77110-5b7a-4349-982a-2bd701c2a7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:59.356534 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db77110-5b7a-4349-982a-2bd701c2a7a3 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:59.356968 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db77110-5b7a-4349-982a-2bd701c2a7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:59.357340 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db77110-5b7a-4349-982a-2bd701c2a7a3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:59.358090 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db77110-5b7a-4349-982a-2bd701c2a7a3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:59.359753 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db77110-5b7a-4349-982a-2bd701c2a7a3 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:59.370404 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-8db77110-5b7a-4349-982a-2bd701c2a7a3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 14:07:59.370404 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:59.370404 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 14:07:59.370404 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 14:07:59.370404 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 14:07:59.370404 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 14:07:59.370404 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 14:07:59.370404 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:07:59.370404 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 14:07:59.370404 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:59.370404 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 14:07:59.370404 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:59.370404 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:59.370404 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:59.370404 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:59.370404 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:59.370404 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:59.371435 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:59.371435 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:59.371435 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:59.371435 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:59.371435 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:59.371435 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:59.371435 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 14:07:59.371435 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 14:07:59.371435 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 14:07:59.371435 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 14:07:59.371435 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:59.371435 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:59.371435 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:59.371435 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:59.371435 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:59.371435 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:59.372285 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:59.372285 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:59.372285 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:59.372285 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:59.372285 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:59.372285 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:59.372285 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:59.372285 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:59.372285 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:59.372285 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:59.372285 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 14:07:59.372285 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:07:59.372285 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 14:07:59.372285 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:07:59.376789 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 473/921] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Aug 5 14:07:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:07:59.378270 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db77110-5b7a-4349-982a-2bd701c2a7a3 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:59.400091 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6a554-baca-4768-a2b6-a6e2652a9f3a None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:59.428260 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:59.451378 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc3512-98e0-476a-bf58-a7a9d03bf6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:59.452555 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc3512-98e0-476a-bf58-a7a9d03bf6e6 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:59.453009 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc3512-98e0-476a-bf58-a7a9d03bf6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:59.453925 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc3512-98e0-476a-bf58-a7a9d03bf6e6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:59.464838 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc3512-98e0-476a-bf58-a7a9d03bf6e6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:59.467413 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc3512-98e0-476a-bf58-a7a9d03bf6e6 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:59.483862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-cecc3512-98e0-476a-bf58-a7a9d03bf6e6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 14:07:59.483862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:59.483862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:59.483862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:59.483862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:59.483862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:59.483862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:59.483862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:59.483862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:59.483862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:59.483862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:59.483862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:59.483862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 14:07:59.483862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 14:07:59.483862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 14:07:59.483862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 14:07:59.484844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:59.484844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:59.484844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:07:59.484844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:59.484844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:07:59.484844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:07:59.484844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:59.484844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:59.484844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:59.484844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:59.484844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:59.484844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:59.484844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:59.484844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:59.484844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:59.484844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:59.484844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:59.484844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:59.485787 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:59.485787 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:59.485787 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 14:07:59.485787 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 14:07:59.485787 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 14:07:59.485787 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 14:07:59.485787 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 14:07:59.485787 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 14:07:59.485787 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 14:07:59.485787 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:59.488484 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 449/922] 10.222.0.26 () {56 vars in 1117 bytes} [Fri Aug 5 14:07:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:07:59.493430 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc3512-98e0-476a-bf58-a7a9d03bf6e6 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:59.500314 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db77110-5b7a-4349-982a-2bd701c2a7a3 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:59.541480 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:59.547086 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260eecc1-f446-49ee-bbc3-0885dfacd0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:59.547835 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260eecc1-f446-49ee-bbc3-0885dfacd0d5 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:59.548183 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260eecc1-f446-49ee-bbc3-0885dfacd0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:59.548562 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260eecc1-f446-49ee-bbc3-0885dfacd0d5 None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:59.550388 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260eecc1-f446-49ee-bbc3-0885dfacd0d5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:59.550700 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260eecc1-f446-49ee-bbc3-0885dfacd0d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:59.552068 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260eecc1-f446-49ee-bbc3-0885dfacd0d5 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:59.560551 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 474/923] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Aug 5 14:07:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:59.564730 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260eecc1-f446-49ee-bbc3-0885dfacd0d5 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:59.634095 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc3512-98e0-476a-bf58-a7a9d03bf6e6 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:59.673848 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd80c3-e2d3-4e77-a2a5-127cf579346e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:59.675906 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd80c3-e2d3-4e77-a2a5-127cf579346e None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:59.676442 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd80c3-e2d3-4e77-a2a5-127cf579346e None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:59.676950 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd80c3-e2d3-4e77-a2a5-127cf579346e None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:59.678080 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd80c3-e2d3-4e77-a2a5-127cf579346e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:59.678676 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd80c3-e2d3-4e77-a2a5-127cf579346e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:59.680308 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd80c3-e2d3-4e77-a2a5-127cf579346e None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:59.700403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 450/924] 10.222.0.26 () {56 vars in 1149 bytes} [Fri Aug 5 14:07:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:59.705080 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260eecc1-f446-49ee-bbc3-0885dfacd0d5 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:59.706651 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd80c3-e2d3-4e77-a2a5-127cf579346e None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:59.709462 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:59.744586 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095fc7d5-dcbb-4a05-b109-dcb66e5726c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:59.745819 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095fc7d5-dcbb-4a05-b109-dcb66e5726c0 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:59.746313 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095fc7d5-dcbb-4a05-b109-dcb66e5726c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:59.746785 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095fc7d5-dcbb-4a05-b109-dcb66e5726c0 None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:59.747690 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095fc7d5-dcbb-4a05-b109-dcb66e5726c0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:07:59.748184 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095fc7d5-dcbb-4a05-b109-dcb66e5726c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:59.749816 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095fc7d5-dcbb-4a05-b109-dcb66e5726c0 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:59.756800 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 475/925] 10.222.0.26 () {56 vars in 1114 bytes} [Fri Aug 5 14:07:59 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:07:59.762231 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095fc7d5-dcbb-4a05-b109-dcb66e5726c0 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:07:59.825696 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:07:59.842070 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd80c3-e2d3-4e77-a2a5-127cf579346e None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:59.901807 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095fc7d5-dcbb-4a05-b109-dcb66e5726c0 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:07:59.911647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95200e-7514-4aa8-adf4-f15989dbb882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:59.912842 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95200e-7514-4aa8-adf4-f15989dbb882 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:59.913407 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95200e-7514-4aa8-adf4-f15989dbb882 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:59.913967 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95200e-7514-4aa8-adf4-f15989dbb882 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:59.930744 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb95200e-7514-4aa8-adf4-f15989dbb882 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:59.932342 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb95200e-7514-4aa8-adf4-f15989dbb882 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:59.949533 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2be342-9602-4045-b1fc-713c9ce25d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:07:59.950152 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2be342-9602-4045-b1fc-713c9ce25d78 None admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:07:59.950427 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2be342-9602-4045-b1fc-713c9ce25d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:07:59.950694 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2be342-9602-4045-b1fc-713c9ce25d78 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:07:59.951404 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2be342-9602-4045-b1fc-713c9ce25d78 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:07:59.952696 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-bb95200e-7514-4aa8-adf4-f15989dbb882 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:59.952696 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:07:59.952696 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:07:59.952696 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:07:59.952696 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:07:59.952696 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:07:59.952696 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:07:59.952696 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:07:59.952696 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:07:59.952696 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:07:59.952696 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:07:59.952696 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:07:59.952696 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:07:59.952696 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:07:59.952696 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:07:59.953740 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:07:59.953740 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:07:59.953740 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:07:59.953740 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:07:59.953740 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:07:59.953740 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:07:59.953740 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:07:59.953740 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:07:59.953740 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:07:59.953740 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:07:59.953740 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:07:59.953740 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:07:59.953740 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:07:59.953740 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:07:59.953740 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:07:59.953740 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:07:59.953740 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:07:59.954848 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:07:59.954848 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:07:59.954848 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:07:59.954848 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:07:59.955336 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2be342-9602-4045-b1fc-713c9ce25d78 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:07:59.957907 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 451/926] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:07:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:07:59.964107 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95200e-7514-4aa8-adf4-f15989dbb882 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:00.043241 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 476/927] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:07:59 2022] POST /identity/v3/endpoints => generated 346 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:08:00.082645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95200e-7514-4aa8-adf4-f15989dbb882 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:00.127496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc847877-e17d-4719-abbc-eb81de57ec77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:00.128126 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc847877-e17d-4719-abbc-eb81de57ec77 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:00.128402 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc847877-e17d-4719-abbc-eb81de57ec77 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:00.128701 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc847877-e17d-4719-abbc-eb81de57ec77 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:00.129402 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc847877-e17d-4719-abbc-eb81de57ec77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:00.129707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc847877-e17d-4719-abbc-eb81de57ec77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:00.131113 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc847877-e17d-4719-abbc-eb81de57ec77 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:00.141363 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:00.146665 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 452/928] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:07:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:08:00.152144 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2be342-9602-4045-b1fc-713c9ce25d78 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:00.183665 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:00.271331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2be342-9602-4045-b1fc-713c9ce25d78 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:00.307942 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1d0c9-10b0-4546-aa70-1cd59e33e4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:00.308542 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1d0c9-10b0-4546-aa70-1cd59e33e4e5 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:00.308827 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1d0c9-10b0-4546-aa70-1cd59e33e4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:00.309100 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1d0c9-10b0-4546-aa70-1cd59e33e4e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:00.316504 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce1d0c9-10b0-4546-aa70-1cd59e33e4e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:00.318127 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce1d0c9-10b0-4546-aa70-1cd59e33e4e5 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:00.330192 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-5ce1d0c9-10b0-4546-aa70-1cd59e33e4e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:08:00.330192 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:00.330192 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:00.330192 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:00.330192 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:00.330192 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:00.330192 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:00.330192 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:00.330192 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:00.330192 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:00.330192 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:00.330192 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:00.330192 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 14:08:00.330192 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 14:08:00.330192 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 14:08:00.331617 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 14:08:00.331617 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:00.331617 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:00.331617 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:00.331617 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:00.331617 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:00.331617 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:00.331617 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:00.331617 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:00.331617 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:00.331617 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:00.331617 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:00.331617 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:00.331617 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:00.331617 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:00.331617 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:00.331617 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:00.332618 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 14:08:00.332618 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 14:08:00.332618 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:08:00.332618 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:08:00.332618 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:08:00.332618 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:08:00.332618 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:08:00.332618 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:00.332618 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 477/929] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:08:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:08:00.339456 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc847877-e17d-4719-abbc-eb81de57ec77 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:00.444791 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc847877-e17d-4719-abbc-eb81de57ec77 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:00.483953 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b50a3-8ab3-4651-b84b-d0c80b16648f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:00.484982 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b50a3-8ab3-4651-b84b-d0c80b16648f None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:00.485469 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b50a3-8ab3-4651-b84b-d0c80b16648f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:00.485947 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b50a3-8ab3-4651-b84b-d0c80b16648f None admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:00.486945 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b50a3-8ab3-4651-b84b-d0c80b16648f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:00.487454 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b50a3-8ab3-4651-b84b-d0c80b16648f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:00.489134 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b50a3-8ab3-4651-b84b-d0c80b16648f None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:00.492561 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426dc0d-48a4-4190-a7ca-5cac612a42b1 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:00.492847 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426dc0d-48a4-4190-a7ca-5cac612a42b1 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:00.493120 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426dc0d-48a4-4190-a7ca-5cac612a42b1 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:00.499323 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 478/930] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:08:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:08:00.500703 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e2e13e-390d-4519-9b6e-d8e149165e7c None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:00.500986 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e2e13e-390d-4519-9b6e-d8e149165e7c None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:00.501259 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e2e13e-390d-4519-9b6e-d8e149165e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:00.502653 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 453/931] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Aug 5 14:08:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:00.508516 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b50a3-8ab3-4651-b84b-d0c80b16648f None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:00.532577 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-b7e2e13e-390d-4519-9b6e-d8e149165e7c None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:08:00.533838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-b7e2e13e-390d-4519-9b6e-d8e149165e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:08:00.588182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:00.623066 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 479/932] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:08:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 14:08:00.628477 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b50a3-8ab3-4651-b84b-d0c80b16648f None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:00.649203 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e2e13e-390d-4519-9b6e-d8e149165e7c None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:00.664897 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:00.670955 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af501316-6144-47df-ad1a-c90c77f67a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:00.671957 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af501316-6144-47df-ad1a-c90c77f67a2e None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:00.672498 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af501316-6144-47df-ad1a-c90c77f67a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:00.673005 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af501316-6144-47df-ad1a-c90c77f67a2e None admin] PATH_INFO: `/v3/projects/319973e011c54b4ca1f2198c8a5e135b/users/17e3b509a2de4159bf234594573f5e18/roles/5c0fa9dd58a349cba21555779c761f0b` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:00.676177 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af501316-6144-47df-ad1a-c90c77f67a2e None admin] RBAC: Authorizing `identity:create_grant(project_id=319973e011c54b4ca1f2198c8a5e135b, user_id=17e3b509a2de4159bf234594573f5e18, role_id=5c0fa9dd58a349cba21555779c761f0b)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:00.677875 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af501316-6144-47df-ad1a-c90c77f67a2e None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:00.754493 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e2e13e-390d-4519-9b6e-d8e149165e7c None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:00.760704 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 454/933] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Aug 5 14:08:00 2022] PUT /identity/v3/projects/319973e011c54b4ca1f2198c8a5e135b/users/17e3b509a2de4159bf234594573f5e18/roles/5c0fa9dd58a349cba21555779c761f0b => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:08:00.762561 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751b103-0da5-496b-8d3b-8611b5d603a1 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:00.763121 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751b103-0da5-496b-8d3b-8611b5d603a1 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:00.763645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751b103-0da5-496b-8d3b-8611b5d603a1 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:00.765713 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 455/934] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:08:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:08:00.774575 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cdc27c-7134-47b3-8b08-627d2b784f81 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:00.775245 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cdc27c-7134-47b3-8b08-627d2b784f81 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:00.775751 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cdc27c-7134-47b3-8b08-627d2b784f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:00.792663 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63146a3-3f2e-4f06-ae03-fc7acd7e6b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:00.793380 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63146a3-3f2e-4f06-ae03-fc7acd7e6b77 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:00.793705 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63146a3-3f2e-4f06-ae03-fc7acd7e6b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:00.793909 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63146a3-3f2e-4f06-ae03-fc7acd7e6b77 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:00.794517 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63146a3-3f2e-4f06-ae03-fc7acd7e6b77 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:00.795931 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63146a3-3f2e-4f06-ae03-fc7acd7e6b77 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:00.799953 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-75cdc27c-7134-47b3-8b08-627d2b784f81 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:08:00.801733 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-75cdc27c-7134-47b3-8b08-627d2b784f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:08:00.806447 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-d63146a3-3f2e-4f06-ae03-fc7acd7e6b77 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 14:08:00.806447 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:00.806447 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 14:08:00.806447 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 14:08:00.806447 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 14:08:00.806447 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 14:08:00.806447 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 14:08:00.806447 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:08:00.806447 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 14:08:00.806447 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:00.806447 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 14:08:00.806447 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:00.806447 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:00.806447 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:00.806447 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:00.806447 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:00.806447 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:00.807527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:00.807527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:00.807527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:00.807527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:00.807527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:00.807527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:00.807527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 14:08:00.807527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 14:08:00.807527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 14:08:00.807527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 14:08:00.807527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:00.807527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:00.807527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:00.807527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:00.807527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:00.807527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:00.808372 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:00.808372 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:00.808372 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:00.808372 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:00.808372 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:00.808372 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:00.808372 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:00.808372 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:00.808372 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:00.808372 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:00.808372 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 14:08:00.808372 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:08:00.808372 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 14:08:00.808372 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:00.808372 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 480/935] 10.222.0.26 () {56 vars in 1126 bytes} [Fri Aug 5 14:08:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:08:00.813928 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63146a3-3f2e-4f06-ae03-fc7acd7e6b77 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:00.893415 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 456/936] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:08:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:08:00.911823 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:00.912916 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cdc27c-7134-47b3-8b08-627d2b784f81 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:00.943165 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63146a3-3f2e-4f06-ae03-fc7acd7e6b77 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:00.989092 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009fa5ea-3525-4dba-8632-05df4e1b2405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:00.989829 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:00.989942 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009fa5ea-3525-4dba-8632-05df4e1b2405 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:00.990277 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009fa5ea-3525-4dba-8632-05df4e1b2405 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:00.990560 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009fa5ea-3525-4dba-8632-05df4e1b2405 None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:00.991413 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009fa5ea-3525-4dba-8632-05df4e1b2405 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:00.992314 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009fa5ea-3525-4dba-8632-05df4e1b2405 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:00.994017 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009fa5ea-3525-4dba-8632-05df4e1b2405 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:01.004557 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 481/937] 10.222.0.26 () {56 vars in 1122 bytes} [Fri Aug 5 14:08:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:01.009482 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009fa5ea-3525-4dba-8632-05df4e1b2405 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:01.047234 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cdc27c-7134-47b3-8b08-627d2b784f81 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:01.084607 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53675329-e697-471f-91d5-1a021cde31dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:01.085344 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53675329-e697-471f-91d5-1a021cde31dd None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:01.085697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53675329-e697-471f-91d5-1a021cde31dd None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:01.086043 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53675329-e697-471f-91d5-1a021cde31dd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:01.086765 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53675329-e697-471f-91d5-1a021cde31dd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:01.088209 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53675329-e697-471f-91d5-1a021cde31dd None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:01.098188 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-53675329-e697-471f-91d5-1a021cde31dd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 14:08:01.098188 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:01.098188 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 14:08:01.098188 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 14:08:01.098188 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 14:08:01.098188 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 14:08:01.098188 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 14:08:01.098188 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:08:01.098188 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 14:08:01.098188 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:08:01.098188 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 14:08:01.098188 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:08:01.098188 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:01.098188 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:01.098188 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:01.098188 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:01.099124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:01.099124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:01.099124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:01.099124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:01.099124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:01.099124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:01.099124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:01.099124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 14:08:01.099124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 14:08:01.099124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 14:08:01.099124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 14:08:01.099124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:01.099124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:01.099124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:01.099124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:01.099124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:01.099947 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:01.099947 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:01.099947 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:01.099947 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:01.099947 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:01.099947 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:01.099947 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:01.099947 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:01.099947 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:01.099947 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:01.099947 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:01.099947 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 14:08:01.099947 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:08:01.099947 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 14:08:01.099947 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:08:01.099947 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 457/938] 10.222.0.26 () {56 vars in 1132 bytes} [Fri Aug 5 14:08:00 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:08:01.109340 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53675329-e697-471f-91d5-1a021cde31dd None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:01.120396 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009fa5ea-3525-4dba-8632-05df4e1b2405 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:01.162781 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b75b7-7e2a-48d5-8a1f-25b8f6c232cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:01.163319 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b75b7-7e2a-48d5-8a1f-25b8f6c232cd None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:01.163952 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b75b7-7e2a-48d5-8a1f-25b8f6c232cd None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:01.164225 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b75b7-7e2a-48d5-8a1f-25b8f6c232cd None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:01.165006 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b75b7-7e2a-48d5-8a1f-25b8f6c232cd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:01.165614 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b75b7-7e2a-48d5-8a1f-25b8f6c232cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:01.167320 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b75b7-7e2a-48d5-8a1f-25b8f6c232cd None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:01.179519 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 482/939] 10.222.0.26 () {56 vars in 1122 bytes} [Fri Aug 5 14:08:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:01.180431 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b75b7-7e2a-48d5-8a1f-25b8f6c232cd None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:01.191238 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:01.247318 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53675329-e697-471f-91d5-1a021cde31dd None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:01.264821 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:01.287214 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f28fb7d-caed-40db-b788-de5038f17a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:01.288088 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28fb7d-caed-40db-b788-de5038f17a8b None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:01.288450 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28fb7d-caed-40db-b788-de5038f17a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:01.288796 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28fb7d-caed-40db-b788-de5038f17a8b None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:01.289642 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f28fb7d-caed-40db-b788-de5038f17a8b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:01.290040 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f28fb7d-caed-40db-b788-de5038f17a8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:01.291464 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f28fb7d-caed-40db-b788-de5038f17a8b None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:01.302643 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 458/940] 10.222.0.26 () {56 vars in 1126 bytes} [Fri Aug 5 14:08:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:01.303356 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f28fb7d-caed-40db-b788-de5038f17a8b None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:01.313685 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b75b7-7e2a-48d5-8a1f-25b8f6c232cd None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:01.361020 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97b118-5024-4497-856f-a7fe1a51bcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:01.362038 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97b118-5024-4497-856f-a7fe1a51bcd8 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:01.363040 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97b118-5024-4497-856f-a7fe1a51bcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:01.363527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97b118-5024-4497-856f-a7fe1a51bcd8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:01.364822 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc97b118-5024-4497-856f-a7fe1a51bcd8 None admin] RBAC: Adding query filter params (interface=public, service_id=30e2ed4e82674ef9b76369bc0fa81610, region_id=RegionOne) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:01.365591 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc97b118-5024-4497-856f-a7fe1a51bcd8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:01.367913 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc97b118-5024-4497-856f-a7fe1a51bcd8 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:01.375868 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 483/941] 10.222.0.26 () {56 vars in 1251 bytes} [Fri Aug 5 14:08:01 2022] GET /identity/v3/endpoints?service_id=30e2ed4e82674ef9b76369bc0fa81610&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:01.381951 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97b118-5024-4497-856f-a7fe1a51bcd8 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:01.427326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f28fb7d-caed-40db-b788-de5038f17a8b None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:01.461863 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:01.466883 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa40bb96-66bb-4d18-af2d-c58bca7177bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:01.467727 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40bb96-66bb-4d18-af2d-c58bca7177bb None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:01.468210 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40bb96-66bb-4d18-af2d-c58bca7177bb None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:01.471692 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40bb96-66bb-4d18-af2d-c58bca7177bb None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:01.471692 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa40bb96-66bb-4d18-af2d-c58bca7177bb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:01.471692 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa40bb96-66bb-4d18-af2d-c58bca7177bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:01.471692 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa40bb96-66bb-4d18-af2d-c58bca7177bb None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:01.482374 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 459/942] 10.222.0.26 () {56 vars in 1126 bytes} [Fri Aug 5 14:08:01 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:08:01.484140 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa40bb96-66bb-4d18-af2d-c58bca7177bb None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:01.510038 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97b118-5024-4497-856f-a7fe1a51bcd8 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:01.537367 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:01.560970 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d8c79-376c-477d-8c35-85626e27ec1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:01.561585 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d8c79-376c-477d-8c35-85626e27ec1f None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:01.561858 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d8c79-376c-477d-8c35-85626e27ec1f None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:01.562125 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d8c79-376c-477d-8c35-85626e27ec1f None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:01.562768 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023d8c79-376c-477d-8c35-85626e27ec1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:01.564112 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023d8c79-376c-477d-8c35-85626e27ec1f None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:01.576891 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 484/943] 10.222.0.26 () {56 vars in 1087 bytes} [Fri Aug 5 14:08:01 2022] GET /identity/v3/services => generated 1497 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:08:01.612171 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa40bb96-66bb-4d18-af2d-c58bca7177bb None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:01.653443 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3feaf4-7456-4806-9593-64c78be7b0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:01.654479 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3feaf4-7456-4806-9593-64c78be7b0ec None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:01.654966 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3feaf4-7456-4806-9593-64c78be7b0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:01.655444 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3feaf4-7456-4806-9593-64c78be7b0ec None admin] PATH_INFO: `/v3/endpoints` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:01.656534 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3feaf4-7456-4806-9593-64c78be7b0ec None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:01.658210 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3feaf4-7456-4806-9593-64c78be7b0ec None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:01.737134 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 460/944] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:08:01 2022] POST /identity/v3/endpoints => generated 347 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:08:02.256956 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25006646-3f36-4745-b41d-56d7a2633178 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:02.257483 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25006646-3f36-4745-b41d-56d7a2633178 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:02.257483 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25006646-3f36-4745-b41d-56d7a2633178 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:02.259082 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 485/945] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:08:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:08:02.264515 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae47ef-b5a0-4c8b-8e17-633061480b6c None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:02.264929 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae47ef-b5a0-4c8b-8e17-633061480b6c None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:02.265100 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae47ef-b5a0-4c8b-8e17-633061480b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:02.285679 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-2fae47ef-b5a0-4c8b-8e17-633061480b6c None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:08:02.286876 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-2fae47ef-b5a0-4c8b-8e17-633061480b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:08:02.364526 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 461/946] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:08:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:08:02.383781 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25006646-3f36-4745-b41d-56d7a2633178 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:02.489993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25006646-3f36-4745-b41d-56d7a2633178 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:02.528854 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece595ac-2029-439f-8538-a266f64938d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:02.529721 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece595ac-2029-439f-8538-a266f64938d3 None admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:02.530221 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece595ac-2029-439f-8538-a266f64938d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:02.530712 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece595ac-2029-439f-8538-a266f64938d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:02.531619 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece595ac-2029-439f-8538-a266f64938d3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:02.533932 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece595ac-2029-439f-8538-a266f64938d3 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:02.536831 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: WARNING keystone.server.flask.common [None req-ece595ac-2029-439f-8538-a266f64938d3 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 05 14:08:02.553366 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:02.594421 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.sql.core [None req-ece595ac-2029-439f-8538-a266f64938d3 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 05 14:08:02.594421 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 05 14:08:02.594421 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [parameters: {'id': '50ee86198f9942239b385fa704b1b6e0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 05 14:08:02.594421 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80322) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 05 14:08:02.603370 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-ece595ac-2029-439f-8538-a266f64938d3 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 50ee86198f9942239b385fa704b1b6e0.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 50ee86198f9942239b385fa704b1b6e0. Aug 05 14:08:02.603370 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:02.603370 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 14:08:02.603370 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 14:08:02.603370 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 14:08:02.603370 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 14:08:02.603370 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 14:08:02.603370 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 14:08:02.603370 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 14:08:02.603370 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application conn.query(q) Aug 05 14:08:02.603370 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 14:08:02.603370 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 14:08:02.603370 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 14:08:02.611533 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application result.read() Aug 05 14:08:02.611533 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 14:08:02.611533 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 14:08:02.611533 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 14:08:02.611533 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 14:08:02.611533 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 14:08:02.611533 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 14:08:02.611533 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 14:08:02.611533 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 14:08:02.611533 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 05 14:08:02.611533 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:02.611533 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 05 14:08:02.611533 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:02.611533 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:02.611533 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 05 14:08:02.611533 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 05 14:08:02.611533 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 05 14:08:02.611533 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 05 14:08:02.612531 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 14:08:02.612531 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application next(self.gen) Aug 05 14:08:02.612531 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 05 14:08:02.612531 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application yield resource Aug 05 14:08:02.612531 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 14:08:02.612531 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application next(self.gen) Aug 05 14:08:02.612531 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 05 14:08:02.612531 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self.session.rollback() Aug 05 14:08:02.612531 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 05 14:08:02.612531 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self.force_reraise() Aug 05 14:08:02.612531 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 05 14:08:02.612531 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise self.value Aug 05 14:08:02.612531 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 05 14:08:02.612531 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 05 14:08:02.612531 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 05 14:08:02.612531 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application session.commit() Aug 05 14:08:02.612531 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 05 14:08:02.613433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 05 14:08:02.613433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 05 14:08:02.613433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self._prepare_impl() Aug 05 14:08:02.613433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 05 14:08:02.613433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self.session.flush() Aug 05 14:08:02.613433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 05 14:08:02.613433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self._flush(objects) Aug 05 14:08:02.613433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 05 14:08:02.613433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 05 14:08:02.613433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 05 14:08:02.613433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application compat.raise_( Aug 05 14:08:02.613433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 14:08:02.613433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception Aug 05 14:08:02.613433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 05 14:08:02.613433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application flush_context.execute() Aug 05 14:08:02.613433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 05 14:08:02.613433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rec.execute(self) Aug 05 14:08:02.614333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 05 14:08:02.614333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 05 14:08:02.614333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 05 14:08:02.614333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 05 14:08:02.614333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 05 14:08:02.614333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 05 14:08:02.614333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 05 14:08:02.614333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 05 14:08:02.614333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 05 14:08:02.614333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 05 14:08:02.614333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 05 14:08:02.614333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 05 14:08:02.614333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 05 14:08:02.614333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 05 14:08:02.614333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 05 14:08:02.615123 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 05 14:08:02.615123 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 14:08:02.615123 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception Aug 05 14:08:02.615123 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 14:08:02.615123 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 14:08:02.615123 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 14:08:02.615123 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 14:08:02.615123 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 14:08:02.615123 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 14:08:02.615123 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 14:08:02.615123 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application conn.query(q) Aug 05 14:08:02.615123 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 14:08:02.615123 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 14:08:02.615123 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 14:08:02.615123 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application result.read() Aug 05 14:08:02.615123 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 14:08:02.615123 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 14:08:02.615994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 14:08:02.615994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 14:08:02.615994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 14:08:02.615994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 14:08:02.615994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 14:08:02.615994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 14:08:02.615994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 05 14:08:02.615994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 05 14:08:02.615994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [parameters: {'id': '50ee86198f9942239b385fa704b1b6e0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 05 14:08:02.615994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 05 14:08:02.615994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:02.615994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 14:08:02.615994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:02.615994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:02.615994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 05 14:08:02.621003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 05 14:08:02.621003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 05 14:08:02.621003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 05 14:08:02.621003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 05 14:08:02.621003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:02.621003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 14:08:02.621003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:02.621003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:02.621003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:02.621003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:02.621003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:02.621003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:02.621003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:02.621003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:02.621003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:02.621003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:02.621003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:02.622122 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:02.622122 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 05 14:08:02.622122 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 05 14:08:02.622122 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:02.622122 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:02.622122 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 05 14:08:02.622122 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 05 14:08:02.622122 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 50ee86198f9942239b385fa704b1b6e0. Aug 05 14:08:02.622122 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:02.622122 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 486/947] 10.222.0.26 () {60 vars in 1140 bytes} [Fri Aug 5 14:08:02 2022] POST /identity/v3/projects => generated 262 bytes in 222 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:08:02.622805 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fae47ef-b5a0-4c8b-8e17-633061480b6c None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:02.651223 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:02.724035 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fae47ef-b5a0-4c8b-8e17-633061480b6c None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:02.761926 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1804111-324b-488d-a507-2b0a085d7ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:02.762574 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1804111-324b-488d-a507-2b0a085d7ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:02.762851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1804111-324b-488d-a507-2b0a085d7ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:02.763125 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1804111-324b-488d-a507-2b0a085d7ad6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:02.770944 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1804111-324b-488d-a507-2b0a085d7ad6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:02.772475 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1804111-324b-488d-a507-2b0a085d7ad6 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:02.784816 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-b1804111-324b-488d-a507-2b0a085d7ad6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:08:02.784816 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:02.784816 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:02.784816 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:02.784816 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:02.784816 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:02.784816 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:02.784816 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:02.784816 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:02.784816 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:02.784816 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:02.784816 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:02.784816 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 14:08:02.784816 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 14:08:02.784816 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 14:08:02.786822 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 14:08:02.786822 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:02.786822 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:02.786822 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:02.786822 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:02.786822 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:02.786822 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:02.786822 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:02.786822 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:02.786822 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:02.786822 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:02.786822 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:02.786822 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:02.786822 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:02.786822 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:02.786822 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:02.786822 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:02.788937 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 14:08:02.788937 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 14:08:02.788937 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:08:02.788937 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:08:02.788937 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:08:02.788937 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:08:02.788937 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:08:02.788937 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:08:02.792934 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 462/948] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:08:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:08:02.793144 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece595ac-2029-439f-8538-a266f64938d3 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:02.903158 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece595ac-2029-439f-8538-a266f64938d3 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:02.930266 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1220c43-ad5b-46a7-abd4-ba02197d619b None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:02.930481 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1220c43-ad5b-46a7-abd4-ba02197d619b None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:02.930719 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1220c43-ad5b-46a7-abd4-ba02197d619b None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:02.933381 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 463/949] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:08:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:08:02.937995 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b591f-ea3d-46a2-ad14-6cd580d4e77b None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:02.938279 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b591f-ea3d-46a2-ad14-6cd580d4e77b None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:02.938550 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b591f-ea3d-46a2-ad14-6cd580d4e77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:02.942696 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc07ad9-c7d7-4b66-80b8-76ab25be2f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:02.943598 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc07ad9-c7d7-4b66-80b8-76ab25be2f8b None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:02.944124 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc07ad9-c7d7-4b66-80b8-76ab25be2f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:02.944632 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc07ad9-c7d7-4b66-80b8-76ab25be2f8b None admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:02.945773 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc07ad9-c7d7-4b66-80b8-76ab25be2f8b None admin] RBAC: Adding query filter params (name=service) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:02.946282 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc07ad9-c7d7-4b66-80b8-76ab25be2f8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:02.948113 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc07ad9-c7d7-4b66-80b8-76ab25be2f8b None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:02.958873 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-4e9b591f-ea3d-46a2-ad14-6cd580d4e77b None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:08:02.960710 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-4e9b591f-ea3d-46a2-ad14-6cd580d4e77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:08:02.977115 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 487/950] 10.222.0.26 () {56 vars in 1112 bytes} [Fri Aug 5 14:08:02 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:03.048142 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 464/951] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:08:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 14:08:03.071534 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc07ad9-c7d7-4b66-80b8-76ab25be2f8b None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:03.129995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:03.181620 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc07ad9-c7d7-4b66-80b8-76ab25be2f8b None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:03.218690 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a396534-a04e-45b0-9ced-283e7c20b0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:03.219341 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a396534-a04e-45b0-9ced-283e7c20b0fa None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:03.219618 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a396534-a04e-45b0-9ced-283e7c20b0fa None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:03.219893 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a396534-a04e-45b0-9ced-283e7c20b0fa None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:03.231229 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a396534-a04e-45b0-9ced-283e7c20b0fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:08:03.236331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-7a396534-a04e-45b0-9ced-283e7c20b0fa None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 14:08:03.236331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:03.236331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:03.236331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:03.236331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:03.236331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:03.236331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:03.236331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:03.236331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:03.236331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:03.236331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:03.236331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:03.236331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:08:03.236331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:08:03.236331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:08:03.236331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:08:03.237436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:08:03.237436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:08:03.237436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:08:03.237436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:08:03.237436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:03.237436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:03.237436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:03.237436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:03.237436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:03.237436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:03.237436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:03.237436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:03.237436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:03.237436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:03.237436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:03.237436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:03.238719 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:03.238719 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:03.238719 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:03.238719 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:03.238719 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:08:03.238719 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:08:03.238719 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:08:03.238719 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:08:03.238719 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:08:03.238719 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:08:03.238719 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 14:08:03.238719 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:03.238719 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 488/952] 10.222.0.26 () {56 vars in 1120 bytes} [Fri Aug 5 14:08:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:08:03.246062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b591f-ea3d-46a2-ad14-6cd580d4e77b None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:03.261848 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:03.353969 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b591f-ea3d-46a2-ad14-6cd580d4e77b None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:03.390424 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee41cdc1-4f54-4f95-8364-b5709780828f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:03.391054 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41cdc1-4f54-4f95-8364-b5709780828f None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:03.391334 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41cdc1-4f54-4f95-8364-b5709780828f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:03.391610 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41cdc1-4f54-4f95-8364-b5709780828f None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:03.392324 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee41cdc1-4f54-4f95-8364-b5709780828f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:03.393302 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee41cdc1-4f54-4f95-8364-b5709780828f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:03.394795 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee41cdc1-4f54-4f95-8364-b5709780828f None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:03.403777 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 465/953] 10.222.0.26 () {56 vars in 1115 bytes} [Fri Aug 5 14:08:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:03.410184 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a396534-a04e-45b0-9ced-283e7c20b0fa None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:03.514457 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a396534-a04e-45b0-9ced-283e7c20b0fa None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:03.529435 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:03.554820 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd537de-9af5-49ba-998d-5c764c29ca2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:03.555469 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd537de-9af5-49ba-998d-5c764c29ca2c None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:03.555744 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd537de-9af5-49ba-998d-5c764c29ca2c None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:03.561740 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd537de-9af5-49ba-998d-5c764c29ca2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:03.567776 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd537de-9af5-49ba-998d-5c764c29ca2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:03.569332 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd537de-9af5-49ba-998d-5c764c29ca2c None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:03.584993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-fdd537de-9af5-49ba-998d-5c764c29ca2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:08:03.584993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:03.584993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:03.584993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:03.584993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:03.584993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:03.584993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:03.584993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:03.584993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:03.584993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:03.584993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:03.584993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:03.584993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:08:03.584993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:08:03.584993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:08:03.586196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:08:03.586196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:03.586196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:03.586196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:03.586196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:03.586196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:03.586196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:03.586196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:03.586196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:03.586196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:03.586196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:03.586196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:03.586196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:03.586196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:03.586196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:03.586196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:03.586196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:03.587100 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:08:03.587100 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:08:03.587100 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:08:03.587100 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:03.590962 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 489/954] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:08:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:08:03.593525 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee41cdc1-4f54-4f95-8364-b5709780828f None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:03.731352 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:03.758200 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb2be9-661e-4538-bd07-14510270b3be None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:03.758512 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb2be9-661e-4538-bd07-14510270b3be None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:03.758749 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb2be9-661e-4538-bd07-14510270b3be None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:03.761563 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 490/955] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:08:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:08:03.765665 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964290d-ae71-4196-a4f3-df2ef3e7dbf3 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:03.765947 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964290d-ae71-4196-a4f3-df2ef3e7dbf3 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:03.766220 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964290d-ae71-4196-a4f3-df2ef3e7dbf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:03.771932 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee41cdc1-4f54-4f95-8364-b5709780828f None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:03.788212 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-8964290d-ae71-4196-a4f3-df2ef3e7dbf3 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:08:03.790325 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-8964290d-ae71-4196-a4f3-df2ef3e7dbf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:08:03.831879 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb9f6e6-57f3-44f4-8369-70a342b10340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:03.832554 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9f6e6-57f3-44f4-8369-70a342b10340 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:03.832838 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9f6e6-57f3-44f4-8369-70a342b10340 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:03.833730 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9f6e6-57f3-44f4-8369-70a342b10340 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:03.834432 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb9f6e6-57f3-44f4-8369-70a342b10340 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:03.834738 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb9f6e6-57f3-44f4-8369-70a342b10340 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:03.836670 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb9f6e6-57f3-44f4-8369-70a342b10340 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:03.854238 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 466/956] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:08:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:03.860284 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb9f6e6-57f3-44f4-8369-70a342b10340 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:03.876433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 491/957] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:08:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:08:03.893268 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8964290d-ae71-4196-a4f3-df2ef3e7dbf3 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:03.979722 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb9f6e6-57f3-44f4-8369-70a342b10340 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:03.988632 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:03.999314 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8964290d-ae71-4196-a4f3-df2ef3e7dbf3 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:04.024174 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd54ef6-ac70-4947-8c9c-1631148ab48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:04.024891 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd54ef6-ac70-4947-8c9c-1631148ab48c None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:04.025234 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd54ef6-ac70-4947-8c9c-1631148ab48c None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:04.025514 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd54ef6-ac70-4947-8c9c-1631148ab48c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:04.028434 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:04.034619 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd54ef6-ac70-4947-8c9c-1631148ab48c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:04.035032 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16330720-1c12-4d27-b2cc-a714a9d6c64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:04.035721 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16330720-1c12-4d27-b2cc-a714a9d6c64f None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:04.035953 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16330720-1c12-4d27-b2cc-a714a9d6c64f None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:04.036110 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd54ef6-ac70-4947-8c9c-1631148ab48c None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:04.036236 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16330720-1c12-4d27-b2cc-a714a9d6c64f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:04.036863 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16330720-1c12-4d27-b2cc-a714a9d6c64f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:04.038208 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16330720-1c12-4d27-b2cc-a714a9d6c64f None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:04.046925 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-16330720-1c12-4d27-b2cc-a714a9d6c64f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 14:08:04.046925 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:04.046925 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 14:08:04.046925 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 14:08:04.046925 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 14:08:04.046925 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 14:08:04.046925 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 14:08:04.046925 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:08:04.046925 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 14:08:04.046925 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:04.046925 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 14:08:04.046925 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:04.046925 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:04.046925 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:04.046925 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:04.046925 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:04.046925 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:04.048021 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:04.048021 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:04.048021 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:04.048021 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:04.048021 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:04.048021 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:04.048021 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 14:08:04.048021 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 14:08:04.048021 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 14:08:04.048021 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 14:08:04.048021 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:04.048021 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:04.048021 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:04.048021 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:04.048021 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:04.048021 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:04.049067 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:04.049067 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:04.049067 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:04.049067 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:04.049067 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:04.049067 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:04.049067 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:04.049067 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:04.049067 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:04.049067 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:04.049067 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 14:08:04.049067 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:08:04.049067 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 14:08:04.049067 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:04.049863 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-9cd54ef6-ac70-4947-8c9c-1631148ab48c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 14:08:04.049863 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:04.049863 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:04.049863 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:04.049863 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:04.049863 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:04.049863 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:04.049863 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:04.049863 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:04.049863 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:04.049863 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:04.049863 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:04.049863 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 14:08:04.049863 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 14:08:04.049863 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 14:08:04.049863 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 14:08:04.050751 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:04.050751 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:04.050751 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:08:04.050751 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:08:04.050751 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:08:04.050751 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:08:04.050751 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:04.050751 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:04.050751 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:04.050751 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:04.050751 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:04.050751 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:04.050751 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:04.050751 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:04.050751 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:04.050751 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:04.050751 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:04.050751 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:04.051707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:04.051707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:04.051707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 14:08:04.051707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 14:08:04.051707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 14:08:04.051707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 14:08:04.051707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 14:08:04.051707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 14:08:04.051707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 14:08:04.051707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:08:04.053383 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 492/958] 10.222.0.26 () {56 vars in 1126 bytes} [Fri Aug 5 14:08:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:08:04.054532 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 467/959] 10.222.0.26 () {56 vars in 1117 bytes} [Fri Aug 5 14:08:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:08:04.054899 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad014c6-150d-4892-ba69-9fed5250face None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:04.055183 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad014c6-150d-4892-ba69-9fed5250face None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:04.055354 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd54ef6-ac70-4947-8c9c-1631148ab48c None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:04.055512 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad014c6-150d-4892-ba69-9fed5250face None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:04.060810 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 493/960] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:08:04.060810 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad014c6-150d-4892-ba69-9fed5250face None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:04.166262 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad014c6-150d-4892-ba69-9fed5250face None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:04.167839 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd54ef6-ac70-4947-8c9c-1631148ab48c None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:04.204365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b1fad0-234a-4a52-8bdf-abab7920648f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:04.205739 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1fad0-234a-4a52-8bdf-abab7920648f None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:04.206024 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1fad0-234a-4a52-8bdf-abab7920648f None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:04.206293 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1fad0-234a-4a52-8bdf-abab7920648f None admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:04.207091 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b1fad0-234a-4a52-8bdf-abab7920648f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:04.207392 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b1fad0-234a-4a52-8bdf-abab7920648f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:04.207990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f3801-6d2f-4a97-81f1-99f86ed682c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:04.208660 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f3801-6d2f-4a97-81f1-99f86ed682c0 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:04.208838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b1fad0-234a-4a52-8bdf-abab7920648f None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:04.208956 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f3801-6d2f-4a97-81f1-99f86ed682c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:04.209196 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f3801-6d2f-4a97-81f1-99f86ed682c0 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:04.209870 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7f3801-6d2f-4a97-81f1-99f86ed682c0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:04.210172 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7f3801-6d2f-4a97-81f1-99f86ed682c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:04.212169 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7f3801-6d2f-4a97-81f1-99f86ed682c0 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:04.216631 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 468/961] 10.222.0.26 () {56 vars in 1122 bytes} [Fri Aug 5 14:08:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:04.217592 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21865f-621c-451a-ba67-3f305e3d1c82 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:04.217872 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21865f-621c-451a-ba67-3f305e3d1c82 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:04.218144 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21865f-621c-451a-ba67-3f305e3d1c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:04.232147 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 494/962] 10.222.0.26 () {56 vars in 1149 bytes} [Fri Aug 5 14:08:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:04.233015 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b1fad0-234a-4a52-8bdf-abab7920648f None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:04.240394 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-2b21865f-621c-451a-ba67-3f305e3d1c82 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:08:04.241568 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-2b21865f-621c-451a-ba67-3f305e3d1c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:08:04.249194 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:04.301521 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:04.313251 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 469/963] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:08:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:08:04.321431 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21865f-621c-451a-ba67-3f305e3d1c82 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:04.339772 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b1fad0-234a-4a52-8bdf-abab7920648f None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:04.376042 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e9dc90-3925-495c-92ee-906a3e826e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:04.376683 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e9dc90-3925-495c-92ee-906a3e826e2e None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:04.377034 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e9dc90-3925-495c-92ee-906a3e826e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:04.377315 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e9dc90-3925-495c-92ee-906a3e826e2e None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:04.377983 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e9dc90-3925-495c-92ee-906a3e826e2e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:04.378286 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e9dc90-3925-495c-92ee-906a3e826e2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:04.379635 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e9dc90-3925-495c-92ee-906a3e826e2e None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:04.389507 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 495/964] 10.222.0.26 () {56 vars in 1122 bytes} [Fri Aug 5 14:08:04 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:04.389507 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e9dc90-3925-495c-92ee-906a3e826e2e None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:04.437137 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21865f-621c-451a-ba67-3f305e3d1c82 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:04.468351 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:04.475130 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e84ab-b7a1-44b4-b217-5cf373c82f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:04.495996 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e84ab-b7a1-44b4-b217-5cf373c82f17 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:04.495996 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e84ab-b7a1-44b4-b217-5cf373c82f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:04.495996 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e84ab-b7a1-44b4-b217-5cf373c82f17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:04.496277 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e9dc90-3925-495c-92ee-906a3e826e2e None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:04.537611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b7e078-61d9-4e24-a3b5-ffbcdf2de29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:04.538248 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b7e078-61d9-4e24-a3b5-ffbcdf2de29d None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:04.538523 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b7e078-61d9-4e24-a3b5-ffbcdf2de29d None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:04.538795 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b7e078-61d9-4e24-a3b5-ffbcdf2de29d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:04.539437 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b7e078-61d9-4e24-a3b5-ffbcdf2de29d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:04.540835 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b7e078-61d9-4e24-a3b5-ffbcdf2de29d None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:04.541290 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:04.550441 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-d3b7e078-61d9-4e24-a3b5-ffbcdf2de29d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 14:08:04.550441 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:04.550441 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 14:08:04.550441 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 14:08:04.550441 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 14:08:04.550441 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 14:08:04.550441 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 14:08:04.550441 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:08:04.550441 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 14:08:04.550441 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:04.550441 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 14:08:04.550441 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:04.550441 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:04.550441 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:04.550441 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:04.550441 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:04.550441 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:04.557330 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:04.557330 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:04.557330 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:04.557330 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:04.557330 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:04.557330 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:04.557330 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 14:08:04.557330 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 14:08:04.557330 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 14:08:04.557330 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 14:08:04.557330 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:04.557330 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:04.557330 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:04.557330 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:04.557330 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:04.557330 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:04.558311 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:04.558311 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:04.558311 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:04.558311 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:04.558311 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:04.558311 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:04.558311 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:04.558311 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:04.558311 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:04.558311 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:04.558311 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 14:08:04.558311 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:08:04.558311 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 14:08:04.558311 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:04.558311 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 496/965] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:08:04 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:08:04.559106 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b7e078-61d9-4e24-a3b5-ffbcdf2de29d None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:04.559182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8e84ab-b7a1-44b4-b217-5cf373c82f17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:04.559182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8e84ab-b7a1-44b4-b217-5cf373c82f17 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:04.664011 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b7e078-61d9-4e24-a3b5-ffbcdf2de29d None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:04.666516 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 470/966] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:08:04 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 14:08:04.667344 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e84ab-b7a1-44b4-b217-5cf373c82f17 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:04.704895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c08d3-ca28-4d1c-b456-a7cc32f16efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:04.706066 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c08d3-ca28-4d1c-b456-a7cc32f16efa None admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:04.706382 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c08d3-ca28-4d1c-b456-a7cc32f16efa None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:04.706647 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c08d3-ca28-4d1c-b456-a7cc32f16efa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:04.707515 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c08d3-ca28-4d1c-b456-a7cc32f16efa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:04.709685 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c08d3-ca28-4d1c-b456-a7cc32f16efa None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:04.764257 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:04.769848 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:04.774715 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e84ab-b7a1-44b4-b217-5cf373c82f17 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:04.784934 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 497/967] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:08:04 2022] POST /identity/v3/endpoints => generated 349 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:08:04.785936 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c08d3-ca28-4d1c-b456-a7cc32f16efa None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:04.833719 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a38b1-a959-444d-8441-dbeede759560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:04.834542 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a38b1-a959-444d-8441-dbeede759560 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:04.834819 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a38b1-a959-444d-8441-dbeede759560 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:04.835090 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a38b1-a959-444d-8441-dbeede759560 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:04.836264 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607a38b1-a959-444d-8441-dbeede759560 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:04.836636 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607a38b1-a959-444d-8441-dbeede759560 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:04.838377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607a38b1-a959-444d-8441-dbeede759560 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:04.848594 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 471/968] 10.222.0.26 () {56 vars in 1112 bytes} [Fri Aug 5 14:08:04 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:04.852283 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a38b1-a959-444d-8441-dbeede759560 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:04.929873 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c08d3-ca28-4d1c-b456-a7cc32f16efa None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:04.968770 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e7ca2-feb1-417c-9e5e-423d70e6f03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:04.969558 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e7ca2-feb1-417c-9e5e-423d70e6f03a None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:04.970022 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e7ca2-feb1-417c-9e5e-423d70e6f03a None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:04.970160 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e7ca2-feb1-417c-9e5e-423d70e6f03a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:04.975083 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a38b1-a959-444d-8441-dbeede759560 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:04.986887 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e7ca2-feb1-417c-9e5e-423d70e6f03a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:04.989003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e7ca2-feb1-417c-9e5e-423d70e6f03a None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:05.005885 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-9a5e7ca2-feb1-417c-9e5e-423d70e6f03a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:08:05.005885 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:05.005885 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:05.005885 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:05.005885 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:05.005885 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:05.005885 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:05.005885 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:05.005885 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:05.005885 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:05.005885 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:05.005885 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:05.005885 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:08:05.005885 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:08:05.005885 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:08:05.006943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:08:05.006943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:05.006943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:05.006943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:05.006943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:05.006943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:05.006943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:05.006943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:05.006943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:05.006943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:05.006943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:05.006943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:05.006943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:05.006943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:05.006943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:05.006943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:05.006943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:05.007905 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:08:05.007905 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:08:05.007905 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:08:05.007905 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:05.012552 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 498/969] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:08:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:08:05.014231 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e7ca2-feb1-417c-9e5e-423d70e6f03a None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:05.015963 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf38b07b-e590-42f1-a9f7-868c54d8f79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:05.016879 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38b07b-e590-42f1-a9f7-868c54d8f79e None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:05.017403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38b07b-e590-42f1-a9f7-868c54d8f79e None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:05.017886 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38b07b-e590-42f1-a9f7-868c54d8f79e None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:05.018839 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf38b07b-e590-42f1-a9f7-868c54d8f79e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:05.019137 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf38b07b-e590-42f1-a9f7-868c54d8f79e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:05.020637 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf38b07b-e590-42f1-a9f7-868c54d8f79e None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:05.030355 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 472/970] 10.222.0.26 () {56 vars in 1112 bytes} [Fri Aug 5 14:08:04 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:05.092820 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:05.112892 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db865e-342f-400c-8b5c-ed9a06764230 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:05.113212 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db865e-342f-400c-8b5c-ed9a06764230 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:05.113445 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db865e-342f-400c-8b5c-ed9a06764230 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:05.115119 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 473/971] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:08:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:08:05.121203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbc0d5-b1a3-4556-80c4-bf874905bda0 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:05.121490 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbc0d5-b1a3-4556-80c4-bf874905bda0 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:05.121759 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbc0d5-b1a3-4556-80c4-bf874905bda0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:05.140840 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-5cdbc0d5-b1a3-4556-80c4-bf874905bda0 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:08:05.141989 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e7ca2-feb1-417c-9e5e-423d70e6f03a None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:05.142199 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5cdbc0d5-b1a3-4556-80c4-bf874905bda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:08:05.152900 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:05.189978 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d886f-51d7-45d8-b0fa-e394cd7b44a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:05.191051 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d886f-51d7-45d8-b0fa-e394cd7b44a0 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:05.191345 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d886f-51d7-45d8-b0fa-e394cd7b44a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:05.191615 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d886f-51d7-45d8-b0fa-e394cd7b44a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:05.192668 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d886f-51d7-45d8-b0fa-e394cd7b44a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:05.193254 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d886f-51d7-45d8-b0fa-e394cd7b44a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:05.194824 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d886f-51d7-45d8-b0fa-e394cd7b44a0 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:05.211004 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 499/972] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:08:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:08:05.216746 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d886f-51d7-45d8-b0fa-e394cd7b44a0 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:05.220087 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 474/973] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:08:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:08:05.247664 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbc0d5-b1a3-4556-80c4-bf874905bda0 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:05.341455 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d886f-51d7-45d8-b0fa-e394cd7b44a0 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:05.356638 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:05.357458 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbc0d5-b1a3-4556-80c4-bf874905bda0 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:05.384951 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b44dd-2fb6-4af8-852c-fd5c36bd5002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:05.385929 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b44dd-2fb6-4af8-852c-fd5c36bd5002 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:05.386401 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b44dd-2fb6-4af8-852c-fd5c36bd5002 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:05.386861 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b44dd-2fb6-4af8-852c-fd5c36bd5002 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:05.395107 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536e2ef-0dfe-4abe-a7ee-3dc2bfa7387c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:05.395468 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b44dd-2fb6-4af8-852c-fd5c36bd5002 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:05.395730 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536e2ef-0dfe-4abe-a7ee-3dc2bfa7387c None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:05.396008 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536e2ef-0dfe-4abe-a7ee-3dc2bfa7387c None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:05.396279 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536e2ef-0dfe-4abe-a7ee-3dc2bfa7387c None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:05.397193 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536e2ef-0dfe-4abe-a7ee-3dc2bfa7387c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:05.397957 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b44dd-2fb6-4af8-852c-fd5c36bd5002 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:05.398604 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536e2ef-0dfe-4abe-a7ee-3dc2bfa7387c None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:05.401155 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: WARNING keystone.server.flask.common [None req-4536e2ef-0dfe-4abe-a7ee-3dc2bfa7387c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 05 14:08:05.410985 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-bd6b44dd-2fb6-4af8-852c-fd5c36bd5002 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:08:05.410985 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:05.410985 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:05.410985 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:05.410985 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:05.410985 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:05.410985 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:05.410985 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:05.410985 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:05.410985 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:05.410985 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:05.410985 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:05.410985 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 14:08:05.410985 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 14:08:05.410985 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 14:08:05.411939 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 14:08:05.411939 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:05.411939 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:05.411939 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:05.411939 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:05.411939 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:05.411939 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:05.411939 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:05.411939 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:05.411939 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:05.411939 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:05.411939 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:05.411939 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:05.411939 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:05.411939 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:05.411939 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:05.411939 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:05.412887 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 14:08:05.412887 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 14:08:05.412887 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:08:05.412887 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:08:05.412887 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:08:05.412887 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:08:05.412887 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:08:05.412887 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:05.415758 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 500/974] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:08:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:08:05.420991 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b44dd-2fb6-4af8-852c-fd5c36bd5002 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:05.440701 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.sql.core [None req-4536e2ef-0dfe-4abe-a7ee-3dc2bfa7387c None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 05 14:08:05.440701 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 05 14:08:05.440701 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [parameters: {'user_id': '268f87c9fe44454bbc7235b9c3ae53f6', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 05 14:08:05.440701 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80321) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 05 14:08:05.464437 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-4536e2ef-0dfe-4abe-a7ee-3dc2bfa7387c None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 05 14:08:05.464437 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:05.464437 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 14:08:05.464437 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 14:08:05.464437 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 14:08:05.464437 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 14:08:05.464437 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 14:08:05.464437 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 14:08:05.464437 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 14:08:05.464437 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application conn.query(q) Aug 05 14:08:05.464437 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 14:08:05.464437 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 14:08:05.464437 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 14:08:05.464437 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application result.read() Aug 05 14:08:05.464437 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 14:08:05.465403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 14:08:05.465403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 14:08:05.465403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 14:08:05.465403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 14:08:05.465403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 14:08:05.465403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 14:08:05.465403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 14:08:05.465403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 05 14:08:05.465403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:08:05.465403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 05 14:08:05.465403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:08:05.465403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:05.465403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 05 14:08:05.465403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 05 14:08:05.465403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 05 14:08:05.465403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 05 14:08:05.465403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 14:08:05.465403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application next(self.gen) Aug 05 14:08:05.466446 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 05 14:08:05.466446 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application yield resource Aug 05 14:08:05.466446 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 14:08:05.466446 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application next(self.gen) Aug 05 14:08:05.466446 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 05 14:08:05.466446 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self.session.rollback() Aug 05 14:08:05.466446 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 05 14:08:05.466446 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self.force_reraise() Aug 05 14:08:05.466446 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 05 14:08:05.466446 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise self.value Aug 05 14:08:05.466446 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 05 14:08:05.466446 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 05 14:08:05.466446 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 05 14:08:05.466446 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application session.commit() Aug 05 14:08:05.466446 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 05 14:08:05.466446 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 05 14:08:05.466446 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 05 14:08:05.467358 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._prepare_impl() Aug 05 14:08:05.467358 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 05 14:08:05.467358 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self.session.flush() Aug 05 14:08:05.467358 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 05 14:08:05.467358 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._flush(objects) Aug 05 14:08:05.467358 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 05 14:08:05.467358 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 05 14:08:05.467358 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 05 14:08:05.467358 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application compat.raise_( Aug 05 14:08:05.467358 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 14:08:05.467358 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception Aug 05 14:08:05.467358 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 05 14:08:05.467358 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application flush_context.execute() Aug 05 14:08:05.467358 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 05 14:08:05.467358 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rec.execute(self) Aug 05 14:08:05.467358 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 05 14:08:05.467358 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 05 14:08:05.468294 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 05 14:08:05.468294 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 05 14:08:05.468294 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 05 14:08:05.468294 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 05 14:08:05.468294 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 05 14:08:05.468294 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 05 14:08:05.468294 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 05 14:08:05.468294 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 05 14:08:05.468294 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 05 14:08:05.468294 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 05 14:08:05.468294 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 05 14:08:05.468294 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 05 14:08:05.468294 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 05 14:08:05.468294 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 05 14:08:05.468294 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 14:08:05.468294 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception Aug 05 14:08:05.469934 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 14:08:05.469934 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 14:08:05.469934 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 14:08:05.469934 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 14:08:05.469934 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 14:08:05.469934 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 14:08:05.469934 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 14:08:05.469934 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application conn.query(q) Aug 05 14:08:05.469934 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 14:08:05.469934 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 14:08:05.469934 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 14:08:05.469934 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application result.read() Aug 05 14:08:05.469934 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 14:08:05.469934 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 14:08:05.469934 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 14:08:05.469934 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 14:08:05.471975 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 14:08:05.471975 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 14:08:05.471975 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 14:08:05.471975 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 14:08:05.471975 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 05 14:08:05.471975 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 05 14:08:05.471975 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [parameters: {'user_id': '268f87c9fe44454bbc7235b9c3ae53f6', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 05 14:08:05.471975 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 05 14:08:05.471975 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:08:05.471975 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 14:08:05.471975 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:08:05.471975 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:05.471975 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:05.471975 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:05.471975 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:05.474244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:05.474244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:05.474244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:05.474244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:05.474244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:05.474244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:05.474244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:05.474244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 05 14:08:05.474244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 05 14:08:05.474244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:05.474244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:05.474244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:08:05.474244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:08:05.474244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:08:05.474244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:08:05.474244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 05 14:08:05.474244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 05 14:08:05.476355 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 05 14:08:05.476355 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 05 14:08:05.476355 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 05 14:08:05.476355 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 05 14:08:05.476355 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 05 14:08:05.476355 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:08:05.477554 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 475/975] 10.222.0.26 () {60 vars in 1131 bytes} [Fri Aug 5 14:08:05 2022] POST /identity/v3/users => generated 160 bytes in 230 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:08:05.479817 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536e2ef-0dfe-4abe-a7ee-3dc2bfa7387c None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:05.493085 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:05.554543 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b44dd-2fb6-4af8-852c-fd5c36bd5002 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:05.599309 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e30d78-5894-4d09-af72-058b96409c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:05.599999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e30d78-5894-4d09-af72-058b96409c35 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:05.600348 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e30d78-5894-4d09-af72-058b96409c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:05.600807 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e30d78-5894-4d09-af72-058b96409c35 None admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:05.601558 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e30d78-5894-4d09-af72-058b96409c35 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:05.601930 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e30d78-5894-4d09-af72-058b96409c35 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:05.603626 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e30d78-5894-4d09-af72-058b96409c35 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:05.625993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 501/976] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Aug 5 14:08:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:08:05.627325 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e30d78-5894-4d09-af72-058b96409c35 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:05.636200 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536e2ef-0dfe-4abe-a7ee-3dc2bfa7387c None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:05.667567 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:05.672338 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e8626d-8758-47c3-90bf-1781148b675c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:05.673062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8626d-8758-47c3-90bf-1781148b675c None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:05.673339 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8626d-8758-47c3-90bf-1781148b675c None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:05.673614 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8626d-8758-47c3-90bf-1781148b675c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:05.681588 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e8626d-8758-47c3-90bf-1781148b675c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:05.683220 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e8626d-8758-47c3-90bf-1781148b675c None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:05.697292 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-e5e8626d-8758-47c3-90bf-1781148b675c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 14:08:05.697292 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:05.697292 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:05.697292 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:05.697292 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:05.697292 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:05.697292 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:05.697292 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:05.697292 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:05.697292 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:05.697292 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:05.697292 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:05.697292 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 14:08:05.697292 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 14:08:05.697292 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 14:08:05.697292 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 14:08:05.698282 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:05.698282 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:05.698282 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:08:05.698282 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:08:05.698282 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:08:05.698282 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:08:05.698282 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:05.698282 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:05.698282 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:05.698282 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:05.698282 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:05.698282 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:05.698282 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:05.698282 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:05.698282 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:05.698282 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:05.698282 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:05.698282 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:05.699310 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:05.699310 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:05.699310 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 14:08:05.699310 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 14:08:05.699310 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 14:08:05.699310 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 14:08:05.699310 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 14:08:05.699310 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 14:08:05.699310 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 14:08:05.699310 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:08:05.703266 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 476/977] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:08:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:08:05.705075 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e8626d-8758-47c3-90bf-1781148b675c None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:05.748029 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e30d78-5894-4d09-af72-058b96409c35 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:05.786350 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167af8a1-72bb-4d73-927d-a08b8ffdc25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:05.787403 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167af8a1-72bb-4d73-927d-a08b8ffdc25b None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:05.787896 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167af8a1-72bb-4d73-927d-a08b8ffdc25b None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:05.788472 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167af8a1-72bb-4d73-927d-a08b8ffdc25b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:05.789352 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:05.789732 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167af8a1-72bb-4d73-927d-a08b8ffdc25b None admin] RBAC: Adding query filter params (role.id=5c0fa9dd58a349cba21555779c761f0b, scope.project.id=319973e011c54b4ca1f2198c8a5e135b, user.id=17e3b509a2de4159bf234594573f5e18) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:05.790253 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167af8a1-72bb-4d73-927d-a08b8ffdc25b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:05.791938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167af8a1-72bb-4d73-927d-a08b8ffdc25b None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:05.804724 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 502/978] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:08:05 2022] GET /identity/v3/role_assignments?user.id=17e3b509a2de4159bf234594573f5e18&scope.project.id=319973e011c54b4ca1f2198c8a5e135b&role.id=5c0fa9dd58a349cba21555779c761f0b => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:05.830712 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e8626d-8758-47c3-90bf-1781148b675c None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:05.870949 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2249de-6df0-406b-aa72-4246a0f543be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:05.871584 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2249de-6df0-406b-aa72-4246a0f543be None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:05.871865 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2249de-6df0-406b-aa72-4246a0f543be None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:05.872140 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2249de-6df0-406b-aa72-4246a0f543be None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:05.872961 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2249de-6df0-406b-aa72-4246a0f543be None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:05.873264 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2249de-6df0-406b-aa72-4246a0f543be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:05.874720 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2249de-6df0-406b-aa72-4246a0f543be None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:05.917812 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 477/979] 10.222.0.26 () {56 vars in 1101 bytes} [Fri Aug 5 14:08:05 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:06.893184 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c21d71-bef6-4050-a24c-5d4db18f35b2 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:06.893737 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c21d71-bef6-4050-a24c-5d4db18f35b2 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:06.893737 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c21d71-bef6-4050-a24c-5d4db18f35b2 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:06.895217 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 503/980] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:08:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:08:06.900266 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025f9e6-6631-4566-8fd2-871aec7e955a None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:06.900697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025f9e6-6631-4566-8fd2-871aec7e955a None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:06.900860 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025f9e6-6631-4566-8fd2-871aec7e955a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:06.919218 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-4025f9e6-6631-4566-8fd2-871aec7e955a None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:08:06.920744 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-4025f9e6-6631-4566-8fd2-871aec7e955a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:08:06.994858 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 478/981] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:08:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:08:07.021792 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c21d71-bef6-4050-a24c-5d4db18f35b2 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:07.121826 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:07.128944 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195c597-0702-402e-ba43-2153fd4bc16e None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:07.129228 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195c597-0702-402e-ba43-2153fd4bc16e None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:07.129494 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195c597-0702-402e-ba43-2153fd4bc16e None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:07.129665 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c21d71-bef6-4050-a24c-5d4db18f35b2 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:07.131287 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 479/982] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:08:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:08:07.136017 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6e984-074e-4e09-8e9f-a320aa8b7072 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:07.136301 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6e984-074e-4e09-8e9f-a320aa8b7072 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:07.141295 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6e984-074e-4e09-8e9f-a320aa8b7072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:07.154560 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-cec6e984-074e-4e09-8e9f-a320aa8b7072 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:08:07.155738 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-cec6e984-074e-4e09-8e9f-a320aa8b7072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:08:07.167363 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfcf77f-4528-44a2-aafb-0328b0a201c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:07.168035 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfcf77f-4528-44a2-aafb-0328b0a201c7 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:07.168288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfcf77f-4528-44a2-aafb-0328b0a201c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:07.168607 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfcf77f-4528-44a2-aafb-0328b0a201c7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:07.169234 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfcf77f-4528-44a2-aafb-0328b0a201c7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:07.170631 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfcf77f-4528-44a2-aafb-0328b0a201c7 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:07.181063 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-cdfcf77f-4528-44a2-aafb-0328b0a201c7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 14:08:07.181063 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:07.181063 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 14:08:07.181063 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 14:08:07.181063 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 14:08:07.181063 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 14:08:07.181063 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 14:08:07.181063 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:08:07.181063 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 14:08:07.181063 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:07.181063 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 14:08:07.181063 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:07.181063 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:07.181063 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:07.181063 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:07.181063 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:07.181063 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:07.182042 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:07.182042 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:07.182042 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:07.182042 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:07.182042 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:07.182042 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:07.182042 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 14:08:07.182042 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 14:08:07.182042 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 14:08:07.182042 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 14:08:07.182042 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:07.182042 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:07.182042 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:07.182042 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:07.182042 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:07.182042 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:07.182901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:07.182901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:07.182901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:07.182901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:07.182901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:07.182901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:07.182901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:07.182901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:07.182901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:07.182901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:07.182901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 14:08:07.182901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:08:07.182901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 14:08:07.182901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:07.182901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 504/983] 10.222.0.26 () {56 vars in 1126 bytes} [Fri Aug 5 14:08:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:08:07.188019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfcf77f-4528-44a2-aafb-0328b0a201c7 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:07.213270 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:07.233501 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 480/984] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:08:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 14:08:07.256970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6e984-074e-4e09-8e9f-a320aa8b7072 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:07.291057 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfcf77f-4528-44a2-aafb-0328b0a201c7 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:07.328664 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396546c7-314b-44c6-a97b-93f5dcfa5ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:07.329384 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396546c7-314b-44c6-a97b-93f5dcfa5ccc None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:07.329661 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396546c7-314b-44c6-a97b-93f5dcfa5ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:07.330071 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396546c7-314b-44c6-a97b-93f5dcfa5ccc None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:07.330974 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396546c7-314b-44c6-a97b-93f5dcfa5ccc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:07.331287 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396546c7-314b-44c6-a97b-93f5dcfa5ccc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:07.332668 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396546c7-314b-44c6-a97b-93f5dcfa5ccc None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:07.342228 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 505/985] 10.222.0.26 () {56 vars in 1122 bytes} [Fri Aug 5 14:08:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:08:07.343201 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396546c7-314b-44c6-a97b-93f5dcfa5ccc None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:07.364805 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6e984-074e-4e09-8e9f-a320aa8b7072 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:07.371217 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:07.403135 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b29c7-66af-488b-9fe5-37e7c23d02b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:07.403752 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b29c7-66af-488b-9fe5-37e7c23d02b7 None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:07.404106 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b29c7-66af-488b-9fe5-37e7c23d02b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:07.404380 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b29c7-66af-488b-9fe5-37e7c23d02b7 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:07.405231 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6b29c7-66af-488b-9fe5-37e7c23d02b7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:07.406545 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6b29c7-66af-488b-9fe5-37e7c23d02b7 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:07.450631 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396546c7-314b-44c6-a97b-93f5dcfa5ccc None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:07.483804 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13980bee-f205-4ed1-8637-f31eaa840f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:07.484438 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13980bee-f205-4ed1-8637-f31eaa840f3b None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:07.484723 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13980bee-f205-4ed1-8637-f31eaa840f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:07.484998 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13980bee-f205-4ed1-8637-f31eaa840f3b None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:07.485673 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13980bee-f205-4ed1-8637-f31eaa840f3b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:07.485977 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13980bee-f205-4ed1-8637-f31eaa840f3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:07.487965 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13980bee-f205-4ed1-8637-f31eaa840f3b None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:07.492793 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 506/986] 10.222.0.26 () {56 vars in 1122 bytes} [Fri Aug 5 14:08:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:08:07.498820 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13980bee-f205-4ed1-8637-f31eaa840f3b None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:07.511591 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 481/987] 10.222.0.26 () {60 vars in 1141 bytes} [Fri Aug 5 14:08:07 2022] POST /identity/v3/services => generated 242 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:08:07.595035 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:07.611818 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13980bee-f205-4ed1-8637-f31eaa840f3b None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:07.667095 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190664d3-aaad-4a47-bc7e-80dd6dd96f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:07.667644 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190664d3-aaad-4a47-bc7e-80dd6dd96f19 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:07.667921 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190664d3-aaad-4a47-bc7e-80dd6dd96f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:07.668193 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190664d3-aaad-4a47-bc7e-80dd6dd96f19 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:07.669070 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190664d3-aaad-4a47-bc7e-80dd6dd96f19 None admin] RBAC: Adding query filter params (interface=admin, service_id=30e2ed4e82674ef9b76369bc0fa81610, region_id=RegionOne) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:07.669375 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190664d3-aaad-4a47-bc7e-80dd6dd96f19 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:07.670722 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190664d3-aaad-4a47-bc7e-80dd6dd96f19 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:07.677006 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 507/988] 10.222.0.26 () {56 vars in 1249 bytes} [Fri Aug 5 14:08:07 2022] GET /identity/v3/endpoints?service_id=30e2ed4e82674ef9b76369bc0fa81610&interface=admin®ion_id=RegionOne => generated 193 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:08:07.682703 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b29c7-66af-488b-9fe5-37e7c23d02b7 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:07.733788 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:07.789125 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b29c7-66af-488b-9fe5-37e7c23d02b7 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:07.825099 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e446975-128b-45e9-9111-9ce26492a8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:07.825716 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e446975-128b-45e9-9111-9ce26492a8c4 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:07.826042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e446975-128b-45e9-9111-9ce26492a8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:07.826314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e446975-128b-45e9-9111-9ce26492a8c4 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:07.826955 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e446975-128b-45e9-9111-9ce26492a8c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:07.828304 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e446975-128b-45e9-9111-9ce26492a8c4 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:07.840496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 482/989] 10.222.0.26 () {56 vars in 1087 bytes} [Fri Aug 5 14:08:07 2022] GET /identity/v3/services => generated 1727 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:08:07.899368 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc85e453-0191-4166-8de8-76dfce3070e7 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:07.899935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc85e453-0191-4166-8de8-76dfce3070e7 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:07.899935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc85e453-0191-4166-8de8-76dfce3070e7 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:07.901328 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 508/990] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:08:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:08:07.906997 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a180226-4c56-4013-aef1-0d096667e301 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:07.907514 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a180226-4c56-4013-aef1-0d096667e301 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:07.907514 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a180226-4c56-4013-aef1-0d096667e301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:07.927029 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-1a180226-4c56-4013-aef1-0d096667e301 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:08:07.928166 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-1a180226-4c56-4013-aef1-0d096667e301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:08:08.003772 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 483/991] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:08:07 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:08:08.025898 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc85e453-0191-4166-8de8-76dfce3070e7 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:08.130395 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc85e453-0191-4166-8de8-76dfce3070e7 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:08.165089 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b3e5e-fc4d-4356-8572-6c95ca51c839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:08.165726 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b3e5e-fc4d-4356-8572-6c95ca51c839 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:08.166000 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b3e5e-fc4d-4356-8572-6c95ca51c839 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:08.166267 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b3e5e-fc4d-4356-8572-6c95ca51c839 None admin] PATH_INFO: `/v3/services/image` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:08.166885 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b3e5e-fc4d-4356-8572-6c95ca51c839 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:08.168224 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b3e5e-fc4d-4356-8572-6c95ca51c839 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:08.178117 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-038b3e5e-fc4d-4356-8572-6c95ca51c839 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 14:08:08.178117 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:08.178117 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 14:08:08.178117 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 14:08:08.178117 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 14:08:08.178117 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 14:08:08.178117 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 14:08:08.178117 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:08:08.178117 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 14:08:08.178117 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:08.178117 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 14:08:08.178117 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:08.178117 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:08.178117 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:08.178117 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:08.178117 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:08.178117 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:08.179258 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:08.179258 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:08.179258 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:08.179258 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:08.179258 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:08.179258 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:08.179258 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 14:08:08.179258 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 14:08:08.179258 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 14:08:08.179258 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 14:08:08.179258 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:08.179258 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:08.179258 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:08.179258 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:08.179258 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:08.179258 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:08.180895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:08.180895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:08.180895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:08.180895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:08.180895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:08.180895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:08.180895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:08.180895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:08.180895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:08.180895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:08.180895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 14:08:08.180895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:08:08.180895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 14:08:08.180895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:08.181912 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 509/992] 10.222.0.26 () {56 vars in 1105 bytes} [Fri Aug 5 14:08:08 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:08:08.185313 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a180226-4c56-4013-aef1-0d096667e301 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:08.225185 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:08.286694 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a180226-4c56-4013-aef1-0d096667e301 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:08.322616 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfff82b-cfd9-4aeb-bd60-3168b0d16a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:08.323238 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfff82b-cfd9-4aeb-bd60-3168b0d16a65 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:08.323515 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfff82b-cfd9-4aeb-bd60-3168b0d16a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:08.323785 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfff82b-cfd9-4aeb-bd60-3168b0d16a65 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:08.324476 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfff82b-cfd9-4aeb-bd60-3168b0d16a65 None admin] RBAC: Adding query filter params (name=image) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:08.324778 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfff82b-cfd9-4aeb-bd60-3168b0d16a65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:08.326124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfff82b-cfd9-4aeb-bd60-3168b0d16a65 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:08.330884 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 484/993] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:08:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:08.336675 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b3e5e-fc4d-4356-8572-6c95ca51c839 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:08.349440 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:08.436833 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b3e5e-fc4d-4356-8572-6c95ca51c839 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:08.470120 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e44e4-cffa-4134-b047-0724049a601e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:08.470809 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e44e4-cffa-4134-b047-0724049a601e None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:08.471088 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e44e4-cffa-4134-b047-0724049a601e None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:08.471355 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e44e4-cffa-4134-b047-0724049a601e None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:08.472024 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e44e4-cffa-4134-b047-0724049a601e None admin] RBAC: Adding query filter params (type=image) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:08.472323 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e44e4-cffa-4134-b047-0724049a601e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:08.473698 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e44e4-cffa-4134-b047-0724049a601e None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:08.478299 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 510/994] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:08:08 2022] GET /identity/v3/services?type=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:09.545237 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d7cf4-51bc-4e1e-a88a-f2378eb90401 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:09.545705 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d7cf4-51bc-4e1e-a88a-f2378eb90401 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:09.545782 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d7cf4-51bc-4e1e-a88a-f2378eb90401 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:09.547239 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 485/995] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:08:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:08:09.552224 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d39c18-bcc7-498d-9b0f-c1cfbb687b13 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:09.552660 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d39c18-bcc7-498d-9b0f-c1cfbb687b13 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:09.552844 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d39c18-bcc7-498d-9b0f-c1cfbb687b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:09.572289 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-e0d39c18-bcc7-498d-9b0f-c1cfbb687b13 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:08:09.573508 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-e0d39c18-bcc7-498d-9b0f-c1cfbb687b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:08:09.641768 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 511/996] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:08:09 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:08:09.664504 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d7cf4-51bc-4e1e-a88a-f2378eb90401 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:09.764424 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d7cf4-51bc-4e1e-a88a-f2378eb90401 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:09.765934 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:09.799733 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba9a63-0819-4e09-88ab-6c57ecafc9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:09.800297 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba9a63-0819-4e09-88ab-6c57ecafc9d0 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:09.800598 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba9a63-0819-4e09-88ab-6c57ecafc9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:09.800868 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba9a63-0819-4e09-88ab-6c57ecafc9d0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:09.801737 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ba9a63-0819-4e09-88ab-6c57ecafc9d0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:09.803347 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ba9a63-0819-4e09-88ab-6c57ecafc9d0 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:09.812234 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-f4ba9a63-0819-4e09-88ab-6c57ecafc9d0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 14:08:09.812234 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:09.812234 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 14:08:09.812234 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 14:08:09.812234 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 14:08:09.812234 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 14:08:09.812234 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 14:08:09.812234 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:08:09.812234 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 14:08:09.812234 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:08:09.812234 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 14:08:09.812234 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:08:09.812234 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:09.812234 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:09.812234 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:09.812234 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:09.812234 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:09.813351 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:09.813351 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:09.813351 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:09.813351 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:09.813351 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:09.813351 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:09.813351 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 14:08:09.813351 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 14:08:09.813351 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 14:08:09.813351 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 14:08:09.813351 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:09.813351 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:09.813351 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:09.813351 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:09.813351 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:09.813351 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:09.814372 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:09.814372 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:09.814372 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:09.814372 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:09.814372 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:09.814372 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:09.814372 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:09.814372 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:09.814372 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:09.814372 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:09.814372 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 14:08:09.814372 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:08:09.814372 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 14:08:09.814372 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:08:09.816088 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 486/997] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:08:09 2022] GET /identity/v3/services/compute => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:08:09.819333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d39c18-bcc7-498d-9b0f-c1cfbb687b13 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:09.822541 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:09.906458 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52264e9f-1397-470d-8659-3431fcf98553 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:09.906788 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52264e9f-1397-470d-8659-3431fcf98553 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:09.907006 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52264e9f-1397-470d-8659-3431fcf98553 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:09.909142 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 487/998] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:08:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:08:09.913922 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1774c3-5c00-462e-9847-6a48c54bfd13 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:09.914203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1774c3-5c00-462e-9847-6a48c54bfd13 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:09.914482 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1774c3-5c00-462e-9847-6a48c54bfd13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:09.921098 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d39c18-bcc7-498d-9b0f-c1cfbb687b13 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:09.932999 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-eb1774c3-5c00-462e-9847-6a48c54bfd13 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:08:09.934187 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-eb1774c3-5c00-462e-9847-6a48c54bfd13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:08:09.959624 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3f21c-007d-41da-a61a-3dd837b2f7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:09.960234 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3f21c-007d-41da-a61a-3dd837b2f7f0 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:09.960536 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3f21c-007d-41da-a61a-3dd837b2f7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:09.960811 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3f21c-007d-41da-a61a-3dd837b2f7f0 None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:09.961490 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d3f21c-007d-41da-a61a-3dd837b2f7f0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:09.961791 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d3f21c-007d-41da-a61a-3dd837b2f7f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:09.963167 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d3f21c-007d-41da-a61a-3dd837b2f7f0 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:09.968307 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 512/999] 10.222.0.26 () {56 vars in 1112 bytes} [Fri Aug 5 14:08:09 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:08:09.974131 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3f21c-007d-41da-a61a-3dd837b2f7f0 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:10.016785 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 488/1000] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:08:09 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 14:08:10.033538 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1774c3-5c00-462e-9847-6a48c54bfd13 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:10.042700 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:10.082330 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3f21c-007d-41da-a61a-3dd837b2f7f0 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:10.120064 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a43440-7c60-43ab-8fb4-9dee9f71d65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:10.120851 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a43440-7c60-43ab-8fb4-9dee9f71d65a None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:10.121226 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a43440-7c60-43ab-8fb4-9dee9f71d65a None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:10.121591 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a43440-7c60-43ab-8fb4-9dee9f71d65a None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:10.122480 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a43440-7c60-43ab-8fb4-9dee9f71d65a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:10.122898 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a43440-7c60-43ab-8fb4-9dee9f71d65a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:10.124400 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a43440-7c60-43ab-8fb4-9dee9f71d65a None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:10.129739 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 513/1001] 10.222.0.26 () {56 vars in 1112 bytes} [Fri Aug 5 14:08:09 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:10.135541 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a43440-7c60-43ab-8fb4-9dee9f71d65a None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:10.136822 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:10.143170 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1774c3-5c00-462e-9847-6a48c54bfd13 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:10.181506 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2595c19f-79e7-4520-8bfe-983c28b4ed54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:10.182142 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595c19f-79e7-4520-8bfe-983c28b4ed54 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:10.182418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595c19f-79e7-4520-8bfe-983c28b4ed54 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:10.182692 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595c19f-79e7-4520-8bfe-983c28b4ed54 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:10.183317 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2595c19f-79e7-4520-8bfe-983c28b4ed54 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:10.184726 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2595c19f-79e7-4520-8bfe-983c28b4ed54 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:10.194496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-2595c19f-79e7-4520-8bfe-983c28b4ed54 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 14:08:10.194496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:10.194496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 14:08:10.194496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 14:08:10.194496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 14:08:10.194496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 14:08:10.194496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 14:08:10.194496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:08:10.194496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 14:08:10.194496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:08:10.194496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 14:08:10.194496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:08:10.194496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:10.194496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:10.194496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:10.194496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:10.194496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:10.195746 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:10.195746 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:10.195746 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:10.195746 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:10.195746 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:10.195746 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:10.195746 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 14:08:10.195746 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 14:08:10.195746 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 14:08:10.195746 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 14:08:10.195746 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:10.195746 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:10.195746 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:10.195746 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:10.195746 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:10.195746 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:10.196821 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:10.196821 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:10.196821 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:10.196821 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:10.196821 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:10.196821 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:10.196821 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:10.196821 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:10.196821 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:10.196821 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:10.196821 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 14:08:10.196821 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:08:10.196821 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 14:08:10.196821 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:08:10.200654 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 489/1002] 10.222.0.26 () {56 vars in 1126 bytes} [Fri Aug 5 14:08:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:08:10.201479 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2595c19f-79e7-4520-8bfe-983c28b4ed54 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:10.245039 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a43440-7c60-43ab-8fb4-9dee9f71d65a None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:10.275154 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:10.279908 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc762519-e79b-4f9e-bb69-319850495242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:10.280644 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc762519-e79b-4f9e-bb69-319850495242 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:10.280973 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc762519-e79b-4f9e-bb69-319850495242 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:10.283935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc762519-e79b-4f9e-bb69-319850495242 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:10.283935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc762519-e79b-4f9e-bb69-319850495242 None admin] RBAC: Adding query filter params (interface=public, service_id=fc90be06d4a549a7be1fe4e44660968b, region_id=RegionOne) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:10.283935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc762519-e79b-4f9e-bb69-319850495242 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:10.283935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc762519-e79b-4f9e-bb69-319850495242 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:10.293618 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 514/1003] 10.222.0.26 () {56 vars in 1251 bytes} [Fri Aug 5 14:08:10 2022] GET /identity/v3/endpoints?service_id=fc90be06d4a549a7be1fe4e44660968b&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:10.294634 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc762519-e79b-4f9e-bb69-319850495242 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:10.309235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2595c19f-79e7-4520-8bfe-983c28b4ed54 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:10.346430 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeb4615-d73f-4703-b6a2-7e6e03edd740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:10.347104 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb4615-d73f-4703-b6a2-7e6e03edd740 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:10.347402 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb4615-d73f-4703-b6a2-7e6e03edd740 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:10.347657 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb4615-d73f-4703-b6a2-7e6e03edd740 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:10.348333 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeb4615-d73f-4703-b6a2-7e6e03edd740 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:10.348667 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeb4615-d73f-4703-b6a2-7e6e03edd740 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:10.350041 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeb4615-d73f-4703-b6a2-7e6e03edd740 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:10.359741 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 490/1004] 10.222.0.26 () {56 vars in 1122 bytes} [Fri Aug 5 14:08:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:10.360726 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeb4615-d73f-4703-b6a2-7e6e03edd740 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:10.389073 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:10.413793 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc762519-e79b-4f9e-bb69-319850495242 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:10.452905 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63963660-c9e0-4ece-aba9-6be9bb819a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:10.453616 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63963660-c9e0-4ece-aba9-6be9bb819a04 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:10.454078 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63963660-c9e0-4ece-aba9-6be9bb819a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:10.454356 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63963660-c9e0-4ece-aba9-6be9bb819a04 None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:10.455679 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63963660-c9e0-4ece-aba9-6be9bb819a04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:10.457265 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63963660-c9e0-4ece-aba9-6be9bb819a04 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:10.466991 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeb4615-d73f-4703-b6a2-7e6e03edd740 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:10.472883 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 515/1005] 10.222.0.26 () {56 vars in 1087 bytes} [Fri Aug 5 14:08:10 2022] GET /identity/v3/services => generated 1727 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:08:10.510145 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dedd502-eb31-42b3-a9f0-6a878a84a28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:10.510925 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedd502-eb31-42b3-a9f0-6a878a84a28d None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:10.511647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedd502-eb31-42b3-a9f0-6a878a84a28d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:10.511647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedd502-eb31-42b3-a9f0-6a878a84a28d None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:10.512532 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dedd502-eb31-42b3-a9f0-6a878a84a28d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:10.512934 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dedd502-eb31-42b3-a9f0-6a878a84a28d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:10.513171 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2333b0-efa8-4151-b030-3ac4c84750b6 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:10.513459 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2333b0-efa8-4151-b030-3ac4c84750b6 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:10.513740 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2333b0-efa8-4151-b030-3ac4c84750b6 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:10.517894 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dedd502-eb31-42b3-a9f0-6a878a84a28d None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:10.518134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 516/1006] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:08:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:08:10.521376 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45352188-de03-4126-af04-b0f3bed4c543 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:10.521656 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45352188-de03-4126-af04-b0f3bed4c543 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:10.521932 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45352188-de03-4126-af04-b0f3bed4c543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:10.532624 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 491/1007] 10.222.0.26 () {56 vars in 1122 bytes} [Fri Aug 5 14:08:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:10.533769 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dedd502-eb31-42b3-a9f0-6a878a84a28d None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:10.546676 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-45352188-de03-4126-af04-b0f3bed4c543 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:08:10.548310 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-45352188-de03-4126-af04-b0f3bed4c543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:08:10.614052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:10.626261 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 517/1008] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:08:10 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 14:08:10.633607 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:10.647560 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dedd502-eb31-42b3-a9f0-6a878a84a28d None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:10.649374 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45352188-de03-4126-af04-b0f3bed4c543 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:10.684309 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d90585d-0da0-4953-8a5d-9e86043a3e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:10.684955 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90585d-0da0-4953-8a5d-9e86043a3e96 None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:10.685235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90585d-0da0-4953-8a5d-9e86043a3e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:10.685508 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90585d-0da0-4953-8a5d-9e86043a3e96 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:10.686215 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d90585d-0da0-4953-8a5d-9e86043a3e96 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:10.687566 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d90585d-0da0-4953-8a5d-9e86043a3e96 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:10.750860 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45352188-de03-4126-af04-b0f3bed4c543 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:10.760809 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 492/1009] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:08:10 2022] POST /identity/v3/endpoints => generated 325 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:08:10.801559 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b1f4fc-d8d3-4fda-a05c-21e3b46b8955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:10.802193 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1f4fc-d8d3-4fda-a05c-21e3b46b8955 None admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:10.802474 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1f4fc-d8d3-4fda-a05c-21e3b46b8955 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:10.802744 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1f4fc-d8d3-4fda-a05c-21e3b46b8955 None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:10.803439 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b1f4fc-d8d3-4fda-a05c-21e3b46b8955 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:10.804869 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b1f4fc-d8d3-4fda-a05c-21e3b46b8955 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:10.876031 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 518/1010] 10.222.0.26 () {60 vars in 1141 bytes} [Fri Aug 5 14:08:10 2022] POST /identity/v3/services => generated 242 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:08:12.480689 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12db2ac-b0bb-4d51-ab3d-774658a00960 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:12.481244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12db2ac-b0bb-4d51-ab3d-774658a00960 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:12.481244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12db2ac-b0bb-4d51-ab3d-774658a00960 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:12.482595 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 493/1011] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:08:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:08:12.487783 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d04aaa-49d0-4b02-94a5-00d13a7b694d None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:12.488180 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d04aaa-49d0-4b02-94a5-00d13a7b694d None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:12.488345 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d04aaa-49d0-4b02-94a5-00d13a7b694d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:12.507079 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-78d04aaa-49d0-4b02-94a5-00d13a7b694d None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:08:12.508257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-78d04aaa-49d0-4b02-94a5-00d13a7b694d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:08:12.556832 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:12.587618 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 519/1012] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:08:12 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:08:12.610137 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12db2ac-b0bb-4d51-ab3d-774658a00960 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:12.712984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12db2ac-b0bb-4d51-ab3d-774658a00960 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:12.746835 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae864d6d-2ad3-4c7d-80ff-2f9d938d4558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:12.747388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae864d6d-2ad3-4c7d-80ff-2f9d938d4558 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:12.747660 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae864d6d-2ad3-4c7d-80ff-2f9d938d4558 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:12.747926 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae864d6d-2ad3-4c7d-80ff-2f9d938d4558 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:12.748568 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae864d6d-2ad3-4c7d-80ff-2f9d938d4558 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:12.749925 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae864d6d-2ad3-4c7d-80ff-2f9d938d4558 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:12.759526 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-ae864d6d-2ad3-4c7d-80ff-2f9d938d4558 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 14:08:12.759526 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:12.759526 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 14:08:12.759526 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 14:08:12.759526 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 14:08:12.759526 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 14:08:12.759526 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 14:08:12.759526 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:08:12.759526 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 14:08:12.759526 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:08:12.759526 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 14:08:12.759526 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:08:12.759526 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:12.759526 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:12.759526 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:12.759526 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:12.759526 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:12.760798 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:12.760798 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:12.760798 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:12.760798 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:12.760798 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:12.760798 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:12.760798 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 14:08:12.760798 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 14:08:12.760798 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 14:08:12.760798 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 14:08:12.760798 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:12.760798 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:12.760798 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:12.760798 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:12.760798 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:12.760798 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:12.761644 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:12.761644 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:12.761644 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:12.761644 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:12.761644 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:12.761644 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:12.761644 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:12.761644 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:12.761644 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:12.761644 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:12.761644 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 14:08:12.761644 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:08:12.761644 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 14:08:12.761644 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:08:12.764494 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 494/1013] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:08:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:08:12.766959 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d04aaa-49d0-4b02-94a5-00d13a7b694d None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:12.796356 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4d455-ed2c-47b3-b1a9-c3fc3c4d785e None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:12.796685 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4d455-ed2c-47b3-b1a9-c3fc3c4d785e None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:12.796963 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4d455-ed2c-47b3-b1a9-c3fc3c4d785e None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:12.798803 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 495/1014] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:08:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:08:12.803700 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3151a1-9680-475f-9b7c-c60038e46143 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:12.803980 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3151a1-9680-475f-9b7c-c60038e46143 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:12.804256 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3151a1-9680-475f-9b7c-c60038e46143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:12.822875 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-1b3151a1-9680-475f-9b7c-c60038e46143 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:08:12.824068 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-1b3151a1-9680-475f-9b7c-c60038e46143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:08:12.831587 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:12.880345 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d04aaa-49d0-4b02-94a5-00d13a7b694d None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:12.893269 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 496/1015] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:08:12 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 14:08:12.900812 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cdb58-e06a-4b0c-a032-3f7238c5a7a0 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:12.901101 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cdb58-e06a-4b0c-a032-3f7238c5a7a0 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:12.901375 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cdb58-e06a-4b0c-a032-3f7238c5a7a0 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:12.902712 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 497/1016] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:08:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:08:12.907349 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184ce03-6fc9-42c4-8b22-c9d7c7254d08 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:12.907631 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184ce03-6fc9-42c4-8b22-c9d7c7254d08 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:12.907940 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184ce03-6fc9-42c4-8b22-c9d7c7254d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:12.917033 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ccbce-4668-4650-b589-237f600d5c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:12.917657 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ccbce-4668-4650-b589-237f600d5c4d None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:12.917934 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ccbce-4668-4650-b589-237f600d5c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:12.918203 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ccbce-4668-4650-b589-237f600d5c4d None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:12.918881 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ccbce-4668-4650-b589-237f600d5c4d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:12.919180 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ccbce-4668-4650-b589-237f600d5c4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:12.920552 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ccbce-4668-4650-b589-237f600d5c4d None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:12.926030 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 520/1017] 10.222.0.26 () {56 vars in 1112 bytes} [Fri Aug 5 14:08:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:12.926230 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-6184ce03-6fc9-42c4-8b22-c9d7c7254d08 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:08:12.926799 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ccbce-4668-4650-b589-237f600d5c4d None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:12.927505 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-6184ce03-6fc9-42c4-8b22-c9d7c7254d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:08:12.961578 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:12.994531 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 498/1018] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:08:12 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 14:08:12.995345 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6184ce03-6fc9-42c4-8b22-c9d7c7254d08 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:13.026042 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ccbce-4668-4650-b589-237f600d5c4d None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:13.062700 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b9333-cea3-4b3a-a02a-899988c49332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:13.063339 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b9333-cea3-4b3a-a02a-899988c49332 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:13.063616 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b9333-cea3-4b3a-a02a-899988c49332 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:13.063955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b9333-cea3-4b3a-a02a-899988c49332 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:13.064808 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:13.065839 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741b9333-cea3-4b3a-a02a-899988c49332 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:13.067250 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741b9333-cea3-4b3a-a02a-899988c49332 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:13.068881 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 521/1019] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:08:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:13.069635 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b9333-cea3-4b3a-a02a-899988c49332 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:13.097645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6184ce03-6fc9-42c4-8b22-c9d7c7254d08 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:13.131668 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96a0ff-6b1d-4eb3-a7ff-45291328ccbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:13.132301 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96a0ff-6b1d-4eb3-a7ff-45291328ccbe None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:13.132603 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96a0ff-6b1d-4eb3-a7ff-45291328ccbe None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:13.132905 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96a0ff-6b1d-4eb3-a7ff-45291328ccbe None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:13.133602 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf96a0ff-6b1d-4eb3-a7ff-45291328ccbe None admin] RBAC: Adding query filter params (type=compute) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:13.133906 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf96a0ff-6b1d-4eb3-a7ff-45291328ccbe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:13.135265 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf96a0ff-6b1d-4eb3-a7ff-45291328ccbe None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:13.139960 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 499/1020] 10.222.0.26 () {56 vars in 1112 bytes} [Fri Aug 5 14:08:12 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:13.140737 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96a0ff-6b1d-4eb3-a7ff-45291328ccbe None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:13.169948 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b9333-cea3-4b3a-a02a-899988c49332 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:13.172209 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:13.203043 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f5733-1038-44f9-809e-2233b9da2243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:13.203668 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f5733-1038-44f9-809e-2233b9da2243 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:13.203940 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f5733-1038-44f9-809e-2233b9da2243 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:13.204203 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f5733-1038-44f9-809e-2233b9da2243 None admin] PATH_INFO: `/v3/services/image` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:13.205056 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f5733-1038-44f9-809e-2233b9da2243 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:13.206398 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f5733-1038-44f9-809e-2233b9da2243 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:13.216387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-c45f5733-1038-44f9-809e-2233b9da2243 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 14:08:13.216387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:13.216387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 14:08:13.216387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 14:08:13.216387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 14:08:13.216387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 14:08:13.216387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 14:08:13.216387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:08:13.216387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 14:08:13.216387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:13.216387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 14:08:13.216387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:13.216387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:13.216387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:13.216387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:13.216387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:13.216387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:13.217527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:13.217527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:13.217527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:13.217527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:13.217527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:13.217527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:13.217527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 14:08:13.217527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 14:08:13.217527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 14:08:13.217527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 14:08:13.217527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:13.217527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:13.217527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:13.217527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:13.217527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:13.217527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:13.218379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:13.218379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:13.218379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:13.218379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:13.218379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:13.218379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:13.218379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:13.218379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:13.218379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:13.218379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:13.218379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 14:08:13.218379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:08:13.218379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 14:08:13.218379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:13.218379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 522/1021] 10.222.0.26 () {56 vars in 1105 bytes} [Fri Aug 5 14:08:13 2022] GET /identity/v3/services/image => generated 86 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:08:13.219268 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f5733-1038-44f9-809e-2233b9da2243 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:13.242153 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96a0ff-6b1d-4eb3-a7ff-45291328ccbe None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:13.275684 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa789e8b-f41d-49ca-95c8-dec11433c4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:13.276345 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa789e8b-f41d-49ca-95c8-dec11433c4d5 None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:13.276604 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa789e8b-f41d-49ca-95c8-dec11433c4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:13.276876 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa789e8b-f41d-49ca-95c8-dec11433c4d5 None admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:13.277552 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa789e8b-f41d-49ca-95c8-dec11433c4d5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:13.278963 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa789e8b-f41d-49ca-95c8-dec11433c4d5 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:13.285078 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:13.319257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f5733-1038-44f9-809e-2233b9da2243 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:13.350002 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 500/1022] 10.222.0.26 () {60 vars in 1141 bytes} [Fri Aug 5 14:08:13 2022] POST /identity/v3/projects => generated 311 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:08:13.350890 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa789e8b-f41d-49ca-95c8-dec11433c4d5 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:13.352505 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f540c3b-6c98-419c-bff5-4bd787f7b0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:13.353201 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f540c3b-6c98-419c-bff5-4bd787f7b0c4 None admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:13.353750 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f540c3b-6c98-419c-bff5-4bd787f7b0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:13.354098 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f540c3b-6c98-419c-bff5-4bd787f7b0c4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:13.354894 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f540c3b-6c98-419c-bff5-4bd787f7b0c4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:13.356369 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f540c3b-6c98-419c-bff5-4bd787f7b0c4 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:13.444402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 523/1023] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:08:13 2022] POST /identity/v3/endpoints => generated 334 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:08:13.473456 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa789e8b-f41d-49ca-95c8-dec11433c4d5 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:13.525873 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07177244-7463-4ecb-a672-c73b7c6c6669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:13.526574 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07177244-7463-4ecb-a672-c73b7c6c6669 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:13.526799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07177244-7463-4ecb-a672-c73b7c6c6669 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:13.527070 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07177244-7463-4ecb-a672-c73b7c6c6669 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:13.527767 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07177244-7463-4ecb-a672-c73b7c6c6669 None admin] RBAC: Adding query filter params (name=image) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:13.528080 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07177244-7463-4ecb-a672-c73b7c6c6669 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:13.529556 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07177244-7463-4ecb-a672-c73b7c6c6669 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:13.536161 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 501/1024] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:08:13 2022] GET /identity/v3/services?name=image => generated 122 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:13.542365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f540c3b-6c98-419c-bff5-4bd787f7b0c4 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:13.571240 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:13.654595 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f540c3b-6c98-419c-bff5-4bd787f7b0c4 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:13.691299 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951a238-78e9-4d55-a266-97c1d69921d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:13.691918 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951a238-78e9-4d55-a266-97c1d69921d1 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:13.692196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951a238-78e9-4d55-a266-97c1d69921d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:13.692530 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951a238-78e9-4d55-a266-97c1d69921d1 None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:13.693212 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951a238-78e9-4d55-a266-97c1d69921d1 None admin] RBAC: Adding query filter params (type=image) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:13.693518 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951a238-78e9-4d55-a266-97c1d69921d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:13.694881 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951a238-78e9-4d55-a266-97c1d69921d1 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:13.699963 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 524/1025] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:08:13 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:13.706433 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07177244-7463-4ecb-a672-c73b7c6c6669 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:13.759495 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:13.810205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07177244-7463-4ecb-a672-c73b7c6c6669 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:13.847221 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071dcc2c-8d86-4e2c-beaa-ae370da562c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:13.847853 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071dcc2c-8d86-4e2c-beaa-ae370da562c9 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:13.848131 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071dcc2c-8d86-4e2c-beaa-ae370da562c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:13.848403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071dcc2c-8d86-4e2c-beaa-ae370da562c9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:13.849176 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071dcc2c-8d86-4e2c-beaa-ae370da562c9 None admin] RBAC: Adding query filter params (interface=public, service_id=02ca8c286b644f56b6f9fd2d8487b37b, region_id=RegionOne) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:13.849494 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071dcc2c-8d86-4e2c-beaa-ae370da562c9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:13.850899 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071dcc2c-8d86-4e2c-beaa-ae370da562c9 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:13.860165 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 502/1026] 10.222.0.26 () {56 vars in 1251 bytes} [Fri Aug 5 14:08:13 2022] GET /identity/v3/endpoints?service_id=02ca8c286b644f56b6f9fd2d8487b37b&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:08:13.861588 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951a238-78e9-4d55-a266-97c1d69921d1 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:13.955872 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:13.967776 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951a238-78e9-4d55-a266-97c1d69921d1 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:14.000332 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76701b7-1e40-4853-a30b-4b43f00420ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:14.000980 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76701b7-1e40-4853-a30b-4b43f00420ee None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:14.001262 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76701b7-1e40-4853-a30b-4b43f00420ee None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:14.002174 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76701b7-1e40-4853-a30b-4b43f00420ee None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:14.003243 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76701b7-1e40-4853-a30b-4b43f00420ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:14.005115 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76701b7-1e40-4853-a30b-4b43f00420ee None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:14.012355 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 525/1027] 10.222.0.26 () {56 vars in 1087 bytes} [Fri Aug 5 14:08:13 2022] GET /identity/v3/services => generated 1957 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:08:15.439361 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f9003-80ad-447c-b496-814ab615c7a0 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:15.439926 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f9003-80ad-447c-b496-814ab615c7a0 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:15.440010 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f9003-80ad-447c-b496-814ab615c7a0 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:15.441416 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 503/1028] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:08:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:08:15.446802 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbfb3a2-347e-4fee-a4ea-61979e1b15d4 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:15.447243 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbfb3a2-347e-4fee-a4ea-61979e1b15d4 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:15.447322 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbfb3a2-347e-4fee-a4ea-61979e1b15d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:15.465680 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-ccbfb3a2-347e-4fee-a4ea-61979e1b15d4 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:08:15.466850 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-ccbfb3a2-347e-4fee-a4ea-61979e1b15d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:08:15.542711 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 526/1029] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:08:15 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:08:15.568012 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f9003-80ad-447c-b496-814ab615c7a0 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:15.598477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8121a00c-8956-4ea7-b42c-52a8aaaf6526 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:15.598778 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8121a00c-8956-4ea7-b42c-52a8aaaf6526 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:15.599093 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8121a00c-8956-4ea7-b42c-52a8aaaf6526 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:15.600556 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 527/1030] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:08:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:08:15.605856 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa6355-2129-4a43-9e19-c1828da460b9 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:15.605856 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa6355-2129-4a43-9e19-c1828da460b9 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:15.605856 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa6355-2129-4a43-9e19-c1828da460b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:15.624785 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-dcfa6355-2129-4a43-9e19-c1828da460b9 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:08:15.625958 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-dcfa6355-2129-4a43-9e19-c1828da460b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:08:15.681639 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f9003-80ad-447c-b496-814ab615c7a0 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:15.695864 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 528/1031] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:08:15 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 14:08:15.696154 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:15.719533 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa6355-2129-4a43-9e19-c1828da460b9 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:15.721192 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9552b26b-594e-4e45-bf7d-99e264035bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:15.721780 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552b26b-594e-4e45-bf7d-99e264035bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:15.722056 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552b26b-594e-4e45-bf7d-99e264035bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:15.722365 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552b26b-594e-4e45-bf7d-99e264035bd3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:15.723527 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9552b26b-594e-4e45-bf7d-99e264035bd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:15.724987 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9552b26b-594e-4e45-bf7d-99e264035bd3 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:15.726717 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 504/1032] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:08:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:15.732324 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9552b26b-594e-4e45-bf7d-99e264035bd3 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:15.735295 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:15.826440 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa6355-2129-4a43-9e19-c1828da460b9 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:15.838016 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9552b26b-594e-4e45-bf7d-99e264035bd3 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:15.863343 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8066e15-1ee9-4ba6-b501-dcc3f0c5ab5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:15.863985 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8066e15-1ee9-4ba6-b501-dcc3f0c5ab5d None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:15.864263 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8066e15-1ee9-4ba6-b501-dcc3f0c5ab5d None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:15.864589 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8066e15-1ee9-4ba6-b501-dcc3f0c5ab5d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:15.874902 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8066e15-1ee9-4ba6-b501-dcc3f0c5ab5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:08:15.875504 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c762b2d0-5a67-4fa2-a21c-0fe00bcca912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:15.876327 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762b2d0-5a67-4fa2-a21c-0fe00bcca912 None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:15.876807 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762b2d0-5a67-4fa2-a21c-0fe00bcca912 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:15.877249 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762b2d0-5a67-4fa2-a21c-0fe00bcca912 None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:15.878166 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c762b2d0-5a67-4fa2-a21c-0fe00bcca912 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:15.879850 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c762b2d0-5a67-4fa2-a21c-0fe00bcca912 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:15.880180 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-e8066e15-1ee9-4ba6-b501-dcc3f0c5ab5d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 14:08:15.880180 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:15.880180 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:15.880180 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:15.880180 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:15.880180 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:15.880180 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:15.880180 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:15.880180 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:15.880180 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:15.880180 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:15.880180 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:15.880180 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:08:15.880180 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:08:15.880180 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:08:15.880180 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:08:15.881157 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:08:15.881157 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:08:15.881157 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:08:15.881157 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:08:15.881157 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:15.881157 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:15.881157 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:15.881157 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:15.881157 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:15.881157 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:15.881157 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:15.881157 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:15.881157 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:15.881157 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:15.881157 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:15.881157 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:15.882374 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:15.882374 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:15.882374 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:15.882374 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:15.882374 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:08:15.882374 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:08:15.882374 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:08:15.882374 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:08:15.882374 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:08:15.882374 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:08:15.882374 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 14:08:15.882374 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:15.882374 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 529/1033] 10.222.0.26 () {56 vars in 1120 bytes} [Fri Aug 5 14:08:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:08:15.887553 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8066e15-1ee9-4ba6-b501-dcc3f0c5ab5d None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:15.952552 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 505/1034] 10.222.0.26 () {60 vars in 1132 bytes} [Fri Aug 5 14:08:15 2022] POST /identity/v3/users => generated 285 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:08:15.971210 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:16.005500 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8066e15-1ee9-4ba6-b501-dcc3f0c5ab5d None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:16.028987 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ee1ee9-e486-4886-8a8b-ac8a0824cefe None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:16.029281 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ee1ee9-e486-4886-8a8b-ac8a0824cefe None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:16.029561 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ee1ee9-e486-4886-8a8b-ac8a0824cefe None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:16.030930 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 506/1035] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:08:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:08:16.037248 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a530b7-b0de-4009-ab7a-f0415c6b4208 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:16.037803 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a530b7-b0de-4009-ab7a-f0415c6b4208 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:16.038086 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a530b7-b0de-4009-ab7a-f0415c6b4208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:16.047787 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1333e6-aa71-431d-bdd5-26fc875b0f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:16.048660 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1333e6-aa71-431d-bdd5-26fc875b0f34 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:16.049126 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1333e6-aa71-431d-bdd5-26fc875b0f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:16.049405 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1333e6-aa71-431d-bdd5-26fc875b0f34 None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:16.050135 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1333e6-aa71-431d-bdd5-26fc875b0f34 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:16.050444 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1333e6-aa71-431d-bdd5-26fc875b0f34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:16.051855 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1333e6-aa71-431d-bdd5-26fc875b0f34 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:16.056880 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-94a530b7-b0de-4009-ab7a-f0415c6b4208 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:08:16.058136 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-94a530b7-b0de-4009-ab7a-f0415c6b4208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:08:16.061134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 530/1036] 10.222.0.26 () {56 vars in 1115 bytes} [Fri Aug 5 14:08:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:08:16.067316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1333e6-aa71-431d-bdd5-26fc875b0f34 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:16.095917 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:16.139195 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 507/1037] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:08:16 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 14:08:16.162949 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a530b7-b0de-4009-ab7a-f0415c6b4208 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:16.188486 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1333e6-aa71-431d-bdd5-26fc875b0f34 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:16.225625 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a138b94-3c57-4cf4-a09d-fdbacce71985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:16.226352 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a138b94-3c57-4cf4-a09d-fdbacce71985 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:16.226733 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a138b94-3c57-4cf4-a09d-fdbacce71985 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:16.227105 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a138b94-3c57-4cf4-a09d-fdbacce71985 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:16.229283 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:16.241292 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a138b94-3c57-4cf4-a09d-fdbacce71985 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:16.251781 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a138b94-3c57-4cf4-a09d-fdbacce71985 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:16.261312 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-4a138b94-3c57-4cf4-a09d-fdbacce71985 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:08:16.261312 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:16.261312 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:16.261312 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:16.261312 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:16.261312 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:16.261312 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:16.261312 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:16.261312 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:16.261312 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:16.261312 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:16.261312 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:16.261312 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:08:16.261312 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:08:16.261312 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:08:16.262412 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:08:16.262412 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:16.262412 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:16.262412 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:16.262412 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:16.262412 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:16.262412 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:16.262412 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:16.262412 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:16.262412 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:16.262412 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:16.262412 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:16.262412 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:16.262412 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:16.262412 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:16.262412 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:16.262412 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:16.263452 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:08:16.263452 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:08:16.263452 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:08:16.263452 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:16.263452 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 531/1038] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:08:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:08:16.268495 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a138b94-3c57-4cf4-a09d-fdbacce71985 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:16.287086 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a530b7-b0de-4009-ab7a-f0415c6b4208 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:16.330223 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6983f24-9327-4c7e-8bb1-c0c0a41a48ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:16.331173 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6983f24-9327-4c7e-8bb1-c0c0a41a48ed None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:16.331460 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6983f24-9327-4c7e-8bb1-c0c0a41a48ed None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:16.331737 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6983f24-9327-4c7e-8bb1-c0c0a41a48ed None admin] PATH_INFO: `/v3/services/image` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:16.332371 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6983f24-9327-4c7e-8bb1-c0c0a41a48ed None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:16.333861 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6983f24-9327-4c7e-8bb1-c0c0a41a48ed None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:16.343176 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-c6983f24-9327-4c7e-8bb1-c0c0a41a48ed None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 14:08:16.343176 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:16.343176 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 14:08:16.343176 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 14:08:16.343176 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 14:08:16.343176 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 14:08:16.343176 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 14:08:16.343176 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:08:16.343176 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 14:08:16.343176 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:08:16.343176 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 14:08:16.343176 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:08:16.343176 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:16.343176 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:16.343176 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:16.343176 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:16.343176 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:16.344203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:16.344203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:16.344203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:16.344203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:16.344203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:16.344203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:16.344203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 14:08:16.344203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 14:08:16.344203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 14:08:16.344203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 14:08:16.344203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:16.344203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:16.344203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:16.344203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:16.344203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:16.344203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:16.346062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:16.346062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:16.346062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:16.346062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:16.346062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:16.346062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:16.346062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:16.346062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:16.346062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:16.346062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:16.346062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 14:08:16.346062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:08:16.346062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 14:08:16.346062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:08:16.346062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 508/1039] 10.222.0.26 () {56 vars in 1105 bytes} [Fri Aug 5 14:08:16 2022] GET /identity/v3/services/image => generated 86 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:08:16.350716 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6983f24-9327-4c7e-8bb1-c0c0a41a48ed None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:16.381912 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a138b94-3c57-4cf4-a09d-fdbacce71985 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:16.383190 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:16.418150 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c481e1-21ad-4df2-9f32-564c32d30491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:16.418769 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c481e1-21ad-4df2-9f32-564c32d30491 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:16.419042 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c481e1-21ad-4df2-9f32-564c32d30491 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:16.419468 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c481e1-21ad-4df2-9f32-564c32d30491 None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:16.420614 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c481e1-21ad-4df2-9f32-564c32d30491 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:16.420614 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c481e1-21ad-4df2-9f32-564c32d30491 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:16.421815 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c481e1-21ad-4df2-9f32-564c32d30491 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:16.435212 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 532/1040] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:08:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:16.441007 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c481e1-21ad-4df2-9f32-564c32d30491 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:16.457141 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6983f24-9327-4c7e-8bb1-c0c0a41a48ed None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:16.494351 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:16.494840 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff4e5b-3a3b-45b3-a10b-29d697742add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:16.495214 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff4e5b-3a3b-45b3-a10b-29d697742add None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:16.495520 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff4e5b-3a3b-45b3-a10b-29d697742add None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:16.495794 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff4e5b-3a3b-45b3-a10b-29d697742add None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:16.496505 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff4e5b-3a3b-45b3-a10b-29d697742add None admin] RBAC: Adding query filter params (name=image) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:16.496816 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff4e5b-3a3b-45b3-a10b-29d697742add None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:16.498291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff4e5b-3a3b-45b3-a10b-29d697742add None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:16.503505 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 509/1041] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:08:16 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:08:16.508975 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff4e5b-3a3b-45b3-a10b-29d697742add None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:16.557393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c481e1-21ad-4df2-9f32-564c32d30491 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:16.603998 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c335f36-c679-4754-b8fd-6ac58a182e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:16.604682 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c335f36-c679-4754-b8fd-6ac58a182e31 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:16.604962 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c335f36-c679-4754-b8fd-6ac58a182e31 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:16.605233 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c335f36-c679-4754-b8fd-6ac58a182e31 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:16.611984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:16.616896 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c335f36-c679-4754-b8fd-6ac58a182e31 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:16.618344 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c335f36-c679-4754-b8fd-6ac58a182e31 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:16.619307 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff4e5b-3a3b-45b3-a10b-29d697742add None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:16.631019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-8c335f36-c679-4754-b8fd-6ac58a182e31 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 14:08:16.631019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:16.631019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:16.631019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:16.631019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:16.631019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:16.631019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:16.631019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:16.631019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:16.631019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:16.631019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:16.631019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:16.631019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 14:08:16.631019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 14:08:16.631019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 14:08:16.631019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 14:08:16.635995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:16.635995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:16.635995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:08:16.635995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:08:16.635995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:08:16.635995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:08:16.635995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:16.635995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:16.635995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:16.635995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:16.635995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:16.635995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:16.635995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:16.635995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:16.635995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:16.635995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:16.635995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:16.635995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:16.637004 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:16.637004 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:16.637004 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 14:08:16.637004 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 14:08:16.637004 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 14:08:16.637004 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 14:08:16.637004 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 14:08:16.637004 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 14:08:16.637004 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 14:08:16.637004 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:16.637004 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 533/1042] 10.222.0.26 () {56 vars in 1093 bytes} [Fri Aug 5 14:08:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:08:16.638884 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c335f36-c679-4754-b8fd-6ac58a182e31 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:16.661675 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e03137-a2cd-4b80-ae85-e1b4ecb636e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:16.662517 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e03137-a2cd-4b80-ae85-e1b4ecb636e3 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:16.662973 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e03137-a2cd-4b80-ae85-e1b4ecb636e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:16.663395 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e03137-a2cd-4b80-ae85-e1b4ecb636e3 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:16.664206 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e03137-a2cd-4b80-ae85-e1b4ecb636e3 None admin] RBAC: Adding query filter params (type=image) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:16.664682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e03137-a2cd-4b80-ae85-e1b4ecb636e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:16.666186 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e03137-a2cd-4b80-ae85-e1b4ecb636e3 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:16.676575 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 510/1043] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:08:16 2022] GET /identity/v3/services?type=image => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:08:16.679468 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e03137-a2cd-4b80-ae85-e1b4ecb636e3 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:16.748699 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c335f36-c679-4754-b8fd-6ac58a182e31 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:16.755894 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:16.784105 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c18f7-00dd-4650-b57b-8ad3244b8a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:16.784750 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c18f7-00dd-4650-b57b-8ad3244b8a1b None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:16.785023 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c18f7-00dd-4650-b57b-8ad3244b8a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:16.785290 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c18f7-00dd-4650-b57b-8ad3244b8a1b None admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:16.786083 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9c18f7-00dd-4650-b57b-8ad3244b8a1b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:16.786382 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9c18f7-00dd-4650-b57b-8ad3244b8a1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:16.786558 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e03137-a2cd-4b80-ae85-e1b4ecb636e3 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:16.787794 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9c18f7-00dd-4650-b57b-8ad3244b8a1b None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:16.805365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 534/1044] 10.222.0.26 () {56 vars in 1133 bytes} [Fri Aug 5 14:08:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:16.811014 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c18f7-00dd-4650-b57b-8ad3244b8a1b None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:16.824399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b2fbd0-e207-4969-bca3-9da1c2a99673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:16.825036 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2fbd0-e207-4969-bca3-9da1c2a99673 None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:16.825309 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2fbd0-e207-4969-bca3-9da1c2a99673 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:16.825584 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2fbd0-e207-4969-bca3-9da1c2a99673 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:16.826276 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b2fbd0-e207-4969-bca3-9da1c2a99673 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:16.827614 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b2fbd0-e207-4969-bca3-9da1c2a99673 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:16.920445 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 511/1045] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:08:16 2022] POST /identity/v3/endpoints => generated 327 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:08:16.920675 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c18f7-00dd-4650-b57b-8ad3244b8a1b None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:16.966915 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e768e78d-a5cb-4bca-93e1-1dd28723343d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:16.967557 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768e78d-a5cb-4bca-93e1-1dd28723343d None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:16.967836 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768e78d-a5cb-4bca-93e1-1dd28723343d None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:16.968105 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768e78d-a5cb-4bca-93e1-1dd28723343d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:17.001618 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:17.045315 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e768e78d-a5cb-4bca-93e1-1dd28723343d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:17.046844 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e768e78d-a5cb-4bca-93e1-1dd28723343d None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:17.154719 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 535/1046] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:08:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 05 14:08:17.161595 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b2fbd0-e207-4969-bca3-9da1c2a99673 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:17.178382 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:17.263872 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b2fbd0-e207-4969-bca3-9da1c2a99673 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:17.297974 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7bed8-ae60-46d4-ac7a-addb672f12f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:17.299148 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7bed8-ae60-46d4-ac7a-addb672f12f6 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:17.299423 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7bed8-ae60-46d4-ac7a-addb672f12f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:17.299691 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7bed8-ae60-46d4-ac7a-addb672f12f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:17.309688 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7bed8-ae60-46d4-ac7a-addb672f12f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:17.311115 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7bed8-ae60-46d4-ac7a-addb672f12f6 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:17.326478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-37e7bed8-ae60-46d4-ac7a-addb672f12f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:08:17.326478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:17.326478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:17.326478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:17.326478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:17.326478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:17.326478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:17.326478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:17.326478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:17.326478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:17.326478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:17.326478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:17.326478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:08:17.326478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:08:17.326478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:08:17.327472 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:08:17.327472 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:17.327472 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:17.327472 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:17.327472 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:17.327472 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:17.327472 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:17.327472 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:17.327472 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:17.327472 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:17.327472 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:17.327472 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:17.327472 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:17.327472 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:17.327472 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:17.327472 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:17.327472 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:17.328395 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:08:17.328395 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:08:17.328395 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:08:17.328395 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:08:17.328395 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 512/1047] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:08:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:08:17.334725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e768e78d-a5cb-4bca-93e1-1dd28723343d None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:17.393635 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:17.436003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e768e78d-a5cb-4bca-93e1-1dd28723343d None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:17.470709 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5412a-d2af-4555-8815-f92fced3be76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:17.471330 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5412a-d2af-4555-8815-f92fced3be76 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:17.471610 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5412a-d2af-4555-8815-f92fced3be76 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:17.471884 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5412a-d2af-4555-8815-f92fced3be76 None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:17.472602 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f5412a-d2af-4555-8815-f92fced3be76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:17.472907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f5412a-d2af-4555-8815-f92fced3be76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:17.474265 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f5412a-d2af-4555-8815-f92fced3be76 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:17.490794 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 536/1048] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:08:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:17.492353 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7bed8-ae60-46d4-ac7a-addb672f12f6 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:17.569621 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:17.605645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7bed8-ae60-46d4-ac7a-addb672f12f6 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:17.643130 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13e55f-b069-44a7-bdc6-a6f321e2bf75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:17.644107 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13e55f-b069-44a7-bdc6-a6f321e2bf75 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:17.644566 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13e55f-b069-44a7-bdc6-a6f321e2bf75 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:17.645005 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13e55f-b069-44a7-bdc6-a6f321e2bf75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:17.651958 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec13e55f-b069-44a7-bdc6-a6f321e2bf75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:17.653739 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec13e55f-b069-44a7-bdc6-a6f321e2bf75 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:17.666235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-ec13e55f-b069-44a7-bdc6-a6f321e2bf75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:08:17.666235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:17.666235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:17.666235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:17.666235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:17.666235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:17.666235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:17.666235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:17.666235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:17.666235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:17.666235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:17.666235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:17.666235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 14:08:17.666235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 14:08:17.666235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 14:08:17.667248 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 14:08:17.667248 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:17.667248 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:17.667248 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:17.667248 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:17.667248 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:17.667248 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:17.667248 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:17.667248 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:17.667248 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:17.667248 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:17.667248 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:17.667248 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:17.667248 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:17.667248 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:17.667248 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:17.667248 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:17.668156 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 14:08:17.668156 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 14:08:17.668156 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:08:17.668156 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:08:17.668156 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:08:17.668156 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:08:17.668156 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:08:17.668156 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:08:17.674020 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 513/1049] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:08:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:08:17.675930 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5412a-d2af-4555-8815-f92fced3be76 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:17.781599 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5412a-d2af-4555-8815-f92fced3be76 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:17.801689 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:17.816425 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f1128-f87c-44ad-ab30-7f0ff0136def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:17.817051 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f1128-f87c-44ad-ab30-7f0ff0136def None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:17.817392 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f1128-f87c-44ad-ab30-7f0ff0136def None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:17.817671 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f1128-f87c-44ad-ab30-7f0ff0136def None admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:17.819982 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f1128-f87c-44ad-ab30-7f0ff0136def None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:17.819982 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f1128-f87c-44ad-ab30-7f0ff0136def None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:17.820216 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f1128-f87c-44ad-ab30-7f0ff0136def None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:17.838182 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 537/1050] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Aug 5 14:08:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:17.839495 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13e55f-b069-44a7-bdc6-a6f321e2bf75 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:17.943482 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13e55f-b069-44a7-bdc6-a6f321e2bf75 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:17.969556 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:17.979128 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e0a1cb-d042-45a4-89c2-553b3167b9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:17.979747 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0a1cb-d042-45a4-89c2-553b3167b9a0 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:17.980024 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0a1cb-d042-45a4-89c2-553b3167b9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:17.980294 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0a1cb-d042-45a4-89c2-553b3167b9a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:17.981166 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e0a1cb-d042-45a4-89c2-553b3167b9a0 None admin] RBAC: Adding query filter params (role.id=5c0fa9dd58a349cba21555779c761f0b, scope.project.id=319973e011c54b4ca1f2198c8a5e135b, user.id=5ec4940c99fc4fa8a5bc7cdabc4558fb) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:17.981470 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e0a1cb-d042-45a4-89c2-553b3167b9a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:17.987108 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e0a1cb-d042-45a4-89c2-553b3167b9a0 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:17.992523 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 514/1051] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:08:17 2022] GET /identity/v3/role_assignments?user.id=5ec4940c99fc4fa8a5bc7cdabc4558fb&scope.project.id=319973e011c54b4ca1f2198c8a5e135b&role.id=5c0fa9dd58a349cba21555779c761f0b => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:18.066637 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa73ed-b770-4144-917e-6db8729b5adf None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:18.066841 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa73ed-b770-4144-917e-6db8729b5adf None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:18.067097 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa73ed-b770-4144-917e-6db8729b5adf None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:18.068537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 538/1052] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:08:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:08:18.074222 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3c527-2089-4746-8581-9461b029256e None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:18.074498 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3c527-2089-4746-8581-9461b029256e None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:18.074774 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3c527-2089-4746-8581-9461b029256e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:18.093828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-43e3c527-2089-4746-8581-9461b029256e None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:08:18.095136 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-43e3c527-2089-4746-8581-9461b029256e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:08:18.172461 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 515/1053] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:08:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:08:18.192740 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa73ed-b770-4144-917e-6db8729b5adf None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:18.294314 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa73ed-b770-4144-917e-6db8729b5adf None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:18.329003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7be74-849b-4616-bd95-14a09def1143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:18.330188 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7be74-849b-4616-bd95-14a09def1143 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:18.330466 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7be74-849b-4616-bd95-14a09def1143 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:18.330739 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7be74-849b-4616-bd95-14a09def1143 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:18.341193 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd7be74-849b-4616-bd95-14a09def1143 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:08:18.346601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-7bd7be74-849b-4616-bd95-14a09def1143 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:08:18.346601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:18.346601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:18.346601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:18.346601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:18.346601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:18.346601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:18.346601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:18.346601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:18.346601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:18.346601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:18.346601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:18.346601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:08:18.346601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:08:18.346601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:08:18.346601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:08:18.347534 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:08:18.347534 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:08:18.347534 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:08:18.347534 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:08:18.347534 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:18.347534 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:18.347534 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:18.347534 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:18.347534 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:18.347534 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:18.347534 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:18.347534 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:18.347534 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:18.347534 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:18.347534 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:18.347534 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:18.348389 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:18.348389 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:18.348389 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:18.348389 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:18.348389 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:08:18.348389 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:08:18.348389 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:08:18.348389 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:08:18.348389 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:08:18.348389 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:08:18.348389 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:08:18.348389 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:18.348389 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 539/1054] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Aug 5 14:08:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:08:18.355303 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e3c527-2089-4746-8581-9461b029256e None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:18.459660 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e3c527-2089-4746-8581-9461b029256e None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:18.479980 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:18.494488 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca2220-e4bd-4edc-95ba-3f2ae969f7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:18.495160 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca2220-e4bd-4edc-95ba-3f2ae969f7b6 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:18.500238 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca2220-e4bd-4edc-95ba-3f2ae969f7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:18.500238 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca2220-e4bd-4edc-95ba-3f2ae969f7b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:18.500238 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ca2220-e4bd-4edc-95ba-3f2ae969f7b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:18.500238 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ca2220-e4bd-4edc-95ba-3f2ae969f7b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:18.500238 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ca2220-e4bd-4edc-95ba-3f2ae969f7b6 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:18.507583 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 516/1055] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:08:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:08:18.513156 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7be74-849b-4616-bd95-14a09def1143 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:18.558825 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:18.616424 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7be74-849b-4616-bd95-14a09def1143 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:18.652796 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff974b3c-1e1a-44b7-88d0-38aa826cd7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:18.653437 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff974b3c-1e1a-44b7-88d0-38aa826cd7dd None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:18.653711 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff974b3c-1e1a-44b7-88d0-38aa826cd7dd None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:18.653981 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff974b3c-1e1a-44b7-88d0-38aa826cd7dd None admin] PATH_INFO: `/v3/users/943fee42863743b5b6f0f327b8754726` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:18.667925 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff974b3c-1e1a-44b7-88d0-38aa826cd7dd None admin] RBAC: Authorizing `identity:get_user(user_id=943fee42863743b5b6f0f327b8754726)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:18.669387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff974b3c-1e1a-44b7-88d0-38aa826cd7dd None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:18.675736 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 540/1056] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:08:18 2022] GET /identity/v3/users/943fee42863743b5b6f0f327b8754726 => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:18.677289 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca2220-e4bd-4edc-95ba-3f2ae969f7b6 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:18.778014 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca2220-e4bd-4edc-95ba-3f2ae969f7b6 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:18.810703 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b6244-3ec5-4c4e-974d-cbbc5b1fe6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:18.811315 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b6244-3ec5-4c4e-974d-cbbc5b1fe6da None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:18.811595 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b6244-3ec5-4c4e-974d-cbbc5b1fe6da None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:18.811865 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b6244-3ec5-4c4e-974d-cbbc5b1fe6da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:18.860216 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:18.879353 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18b6244-3ec5-4c4e-974d-cbbc5b1fe6da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:18.880806 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18b6244-3ec5-4c4e-974d-cbbc5b1fe6da None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:18.960046 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1810b11-7458-4cb6-aaea-c3d8ea8e8ad5 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:18.960626 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1810b11-7458-4cb6-aaea-c3d8ea8e8ad5 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:18.960626 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1810b11-7458-4cb6-aaea-c3d8ea8e8ad5 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:18.962078 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 541/1057] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:08:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:08:18.966726 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd9ceb-bc73-42d4-9c78-63b62f5e4d26 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:18.967005 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd9ceb-bc73-42d4-9c78-63b62f5e4d26 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:18.967281 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd9ceb-bc73-42d4-9c78-63b62f5e4d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:18.981925 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 517/1058] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:08:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 14:08:18.985221 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-30fd9ceb-bc73-42d4-9c78-63b62f5e4d26 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:08:18.986923 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-30fd9ceb-bc73-42d4-9c78-63b62f5e4d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:08:18.987780 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b6244-3ec5-4c4e-974d-cbbc5b1fe6da None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:19.058458 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 542/1059] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:08:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 14:08:19.077573 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:19.081424 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd9ceb-bc73-42d4-9c78-63b62f5e4d26 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:19.098594 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b6244-3ec5-4c4e-974d-cbbc5b1fe6da None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:19.117988 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:19.138122 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe197a-65d7-4d4d-9f86-905ffabe2b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:19.138733 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe197a-65d7-4d4d-9f86-905ffabe2b00 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:19.139032 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe197a-65d7-4d4d-9f86-905ffabe2b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:19.139334 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe197a-65d7-4d4d-9f86-905ffabe2b00 None admin] PATH_INFO: `/v3/projects/2a47193e470c44c9988a1094aa4dddc3` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:19.140470 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fe197a-65d7-4d4d-9f86-905ffabe2b00 None admin] RBAC: Authorizing `identity:get_project(project_id=2a47193e470c44c9988a1094aa4dddc3)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:19.142013 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fe197a-65d7-4d4d-9f86-905ffabe2b00 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:19.144540 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 518/1060] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:08:18 2022] GET /identity/v3/projects/2a47193e470c44c9988a1094aa4dddc3 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:08:19.150777 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe197a-65d7-4d4d-9f86-905ffabe2b00 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:19.193955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd9ceb-bc73-42d4-9c78-63b62f5e4d26 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:19.228115 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af34416-b56d-4276-a77b-b4e6c428d350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:19.228763 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af34416-b56d-4276-a77b-b4e6c428d350 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:19.229039 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af34416-b56d-4276-a77b-b4e6c428d350 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:19.229304 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af34416-b56d-4276-a77b-b4e6c428d350 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:19.253260 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe197a-65d7-4d4d-9f86-905ffabe2b00 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:19.288206 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66959b18-9dd4-495b-9be7-cc71e07e07a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:19.288862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66959b18-9dd4-495b-9be7-cc71e07e07a8 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:19.289144 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66959b18-9dd4-495b-9be7-cc71e07e07a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:19.289418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66959b18-9dd4-495b-9be7-cc71e07e07a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:19.290196 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66959b18-9dd4-495b-9be7-cc71e07e07a8 None admin] RBAC: Adding query filter params (role.id=299c794c7acb4a5ca36288c3191aad10, scope.project.id=2a47193e470c44c9988a1094aa4dddc3, user.id=943fee42863743b5b6f0f327b8754726) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:19.290505 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66959b18-9dd4-495b-9be7-cc71e07e07a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:19.291908 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66959b18-9dd4-495b-9be7-cc71e07e07a8 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:19.296785 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af34416-b56d-4276-a77b-b4e6c428d350 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:19.297836 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 519/1061] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:08:19 2022] GET /identity/v3/role_assignments?user.id=943fee42863743b5b6f0f327b8754726&scope.project.id=2a47193e470c44c9988a1094aa4dddc3&role.id=299c794c7acb4a5ca36288c3191aad10 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:19.298416 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af34416-b56d-4276-a77b-b4e6c428d350 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:19.330697 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:19.413331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 543/1062] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:08:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 05 14:08:19.414597 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66959b18-9dd4-495b-9be7-cc71e07e07a8 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:19.432723 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:19.529803 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66959b18-9dd4-495b-9be7-cc71e07e07a8 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:19.564050 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24ff58-c20e-4a44-87ec-47f82520bf16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:19.564715 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24ff58-c20e-4a44-87ec-47f82520bf16 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:19.564989 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24ff58-c20e-4a44-87ec-47f82520bf16 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:19.565255 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24ff58-c20e-4a44-87ec-47f82520bf16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:19.575518 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e24ff58-c20e-4a44-87ec-47f82520bf16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:19.577049 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e24ff58-c20e-4a44-87ec-47f82520bf16 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:19.592399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-0e24ff58-c20e-4a44-87ec-47f82520bf16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:08:19.592399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:19.592399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:19.592399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:19.592399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:19.592399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:19.592399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:19.592399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:19.592399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:19.592399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:19.592399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:19.592399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:19.592399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:08:19.592399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:08:19.592399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:08:19.593377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:08:19.593377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:19.593377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:19.593377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:19.593377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:19.593377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:19.593377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:19.593377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:19.593377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:19.593377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:19.593377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:19.593377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:19.593377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:19.593377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:19.593377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:19.593377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:19.593377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:19.594309 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:08:19.594309 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:08:19.594309 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:08:19.594309 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:08:19.595867 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 520/1063] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:08:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:08:19.600308 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af34416-b56d-4276-a77b-b4e6c428d350 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:19.738066 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af34416-b56d-4276-a77b-b4e6c428d350 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:19.766313 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:19.775322 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab506b-c2d4-435f-a0c2-3963b2b1dd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:19.775893 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab506b-c2d4-435f-a0c2-3963b2b1dd9b None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:19.776169 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab506b-c2d4-435f-a0c2-3963b2b1dd9b None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:19.780973 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab506b-c2d4-435f-a0c2-3963b2b1dd9b None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:19.780973 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab506b-c2d4-435f-a0c2-3963b2b1dd9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:19.780973 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab506b-c2d4-435f-a0c2-3963b2b1dd9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:19.780973 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab506b-c2d4-435f-a0c2-3963b2b1dd9b None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:19.796446 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 544/1064] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:08:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:08:20.042526 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d94e9-330e-4825-a66c-ee6569c75e1a None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:20.042716 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d94e9-330e-4825-a66c-ee6569c75e1a None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:20.042995 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d94e9-330e-4825-a66c-ee6569c75e1a None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:20.044624 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 521/1065] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:08:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:08:20.050395 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabebaf1-53ab-4164-af34-ac5b33b54f7c None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:20.050990 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabebaf1-53ab-4164-af34-ac5b33b54f7c None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:20.050990 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabebaf1-53ab-4164-af34-ac5b33b54f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:20.069431 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-dabebaf1-53ab-4164-af34-ac5b33b54f7c None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:08:20.070605 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-dabebaf1-53ab-4164-af34-ac5b33b54f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:08:20.135795 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 545/1066] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:08:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:08:20.158000 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d94e9-330e-4825-a66c-ee6569c75e1a None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:20.222141 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:20.256781 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d94e9-330e-4825-a66c-ee6569c75e1a None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:20.290257 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe594baa-71dc-406d-8b3e-6b723aabab07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:20.290882 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe594baa-71dc-406d-8b3e-6b723aabab07 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:20.291158 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe594baa-71dc-406d-8b3e-6b723aabab07 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:20.291427 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe594baa-71dc-406d-8b3e-6b723aabab07 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:20.301754 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe594baa-71dc-406d-8b3e-6b723aabab07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:08:20.306584 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-fe594baa-71dc-406d-8b3e-6b723aabab07 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 14:08:20.306584 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:20.306584 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:20.306584 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:20.306584 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:20.306584 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:20.306584 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:20.306584 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:20.306584 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:20.306584 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:20.306584 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:20.306584 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:20.306584 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:08:20.306584 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:08:20.306584 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:08:20.306584 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:08:20.307520 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:08:20.307520 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:08:20.307520 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:08:20.307520 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:08:20.307520 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:20.307520 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:20.307520 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:20.307520 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:20.307520 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:20.307520 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:20.307520 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:20.307520 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:20.307520 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:20.307520 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:20.307520 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:20.307520 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:20.308386 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:20.308386 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:20.308386 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:20.308386 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:20.308386 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:08:20.308386 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:08:20.308386 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:08:20.308386 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:08:20.308386 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:08:20.308386 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:08:20.308386 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 14:08:20.308386 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:08:20.308386 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 522/1067] 10.222.0.26 () {56 vars in 1120 bytes} [Fri Aug 5 14:08:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:08:20.313707 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabebaf1-53ab-4164-af34-ac5b33b54f7c None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:20.411805 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabebaf1-53ab-4164-af34-ac5b33b54f7c None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:20.429594 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:20.446524 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911354e0-86f2-431e-aa64-a04cb0768574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:20.447147 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911354e0-86f2-431e-aa64-a04cb0768574 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:20.447425 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911354e0-86f2-431e-aa64-a04cb0768574 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:20.447726 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911354e0-86f2-431e-aa64-a04cb0768574 None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:20.448467 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911354e0-86f2-431e-aa64-a04cb0768574 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:20.448772 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911354e0-86f2-431e-aa64-a04cb0768574 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:20.450222 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911354e0-86f2-431e-aa64-a04cb0768574 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:20.458420 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 546/1068] 10.222.0.26 () {56 vars in 1115 bytes} [Fri Aug 5 14:08:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:20.464468 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe594baa-71dc-406d-8b3e-6b723aabab07 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:20.564822 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe594baa-71dc-406d-8b3e-6b723aabab07 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:20.594272 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:20.603368 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37240245-6cfb-4635-a9d0-e352493b97fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:20.603989 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37240245-6cfb-4635-a9d0-e352493b97fa None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:20.604263 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37240245-6cfb-4635-a9d0-e352493b97fa None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:20.604580 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37240245-6cfb-4635-a9d0-e352493b97fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:20.616314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37240245-6cfb-4635-a9d0-e352493b97fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:20.617768 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37240245-6cfb-4635-a9d0-e352493b97fa None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:20.632563 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-37240245-6cfb-4635-a9d0-e352493b97fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:08:20.632563 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:20.632563 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:20.632563 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:20.632563 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:20.632563 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:20.632563 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:20.632563 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:20.632563 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:20.632563 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:20.632563 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:20.632563 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:20.632563 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:08:20.632563 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:08:20.632563 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:08:20.633670 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:08:20.633670 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:20.633670 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:20.633670 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:20.633670 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:20.633670 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:20.633670 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:20.633670 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:20.633670 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:20.633670 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:20.633670 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:20.633670 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:20.633670 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:20.633670 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:20.633670 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:20.633670 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:20.633670 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:20.634792 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:08:20.634792 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:08:20.634792 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:08:20.634792 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:08:20.634792 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 523/1069] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:08:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:08:20.639603 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911354e0-86f2-431e-aa64-a04cb0768574 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:20.760129 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911354e0-86f2-431e-aa64-a04cb0768574 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:20.803186 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90549db9-d337-4f42-b2e1-24052da4da45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:20.803803 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90549db9-d337-4f42-b2e1-24052da4da45 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:20.804077 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90549db9-d337-4f42-b2e1-24052da4da45 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:20.804345 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90549db9-d337-4f42-b2e1-24052da4da45 None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:20.805061 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90549db9-d337-4f42-b2e1-24052da4da45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:20.805437 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90549db9-d337-4f42-b2e1-24052da4da45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:20.806793 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90549db9-d337-4f42-b2e1-24052da4da45 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:20.819816 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 547/1070] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:08:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:20.827132 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37240245-6cfb-4635-a9d0-e352493b97fa None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:20.950827 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37240245-6cfb-4635-a9d0-e352493b97fa None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:20.989062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2371dd95-4492-4f32-a3b7-ae17cb1bd17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:20.989702 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2371dd95-4492-4f32-a3b7-ae17cb1bd17e None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:20.989975 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2371dd95-4492-4f32-a3b7-ae17cb1bd17e None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:20.990247 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2371dd95-4492-4f32-a3b7-ae17cb1bd17e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:20.998593 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2371dd95-4492-4f32-a3b7-ae17cb1bd17e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:21.000028 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2371dd95-4492-4f32-a3b7-ae17cb1bd17e None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:21.012150 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-2371dd95-4492-4f32-a3b7-ae17cb1bd17e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 14:08:21.012150 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:21.012150 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:21.012150 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:21.012150 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:21.012150 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:21.012150 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:21.012150 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:21.012150 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:21.012150 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:21.012150 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:21.012150 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:21.012150 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 14:08:21.012150 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 14:08:21.012150 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 14:08:21.012150 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 14:08:21.013396 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:21.013396 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:21.013396 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:08:21.013396 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:08:21.013396 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:08:21.013396 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:08:21.013396 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:21.013396 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:21.013396 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:21.013396 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:21.013396 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:21.013396 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:21.013396 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:21.013396 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:21.013396 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:21.013396 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:21.013396 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:21.013396 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:21.014348 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:21.014348 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:21.014348 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 14:08:21.014348 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 14:08:21.014348 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 14:08:21.014348 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 14:08:21.014348 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 14:08:21.014348 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 14:08:21.014348 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 14:08:21.014348 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:08:21.014348 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 524/1071] 10.222.0.26 () {56 vars in 1093 bytes} [Fri Aug 5 14:08:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:08:21.019823 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90549db9-d337-4f42-b2e1-24052da4da45 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:21.069258 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:21.139042 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90549db9-d337-4f42-b2e1-24052da4da45 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:21.177092 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe4109-6b21-4e6d-825a-61454029d908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:21.177734 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe4109-6b21-4e6d-825a-61454029d908 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:21.178010 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe4109-6b21-4e6d-825a-61454029d908 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:21.178278 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe4109-6b21-4e6d-825a-61454029d908 None admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:21.179255 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfe4109-6b21-4e6d-825a-61454029d908 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:21.179555 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfe4109-6b21-4e6d-825a-61454029d908 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:21.181002 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfe4109-6b21-4e6d-825a-61454029d908 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:21.198798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 548/1072] 10.222.0.26 () {56 vars in 1133 bytes} [Fri Aug 5 14:08:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:08:21.205061 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2371dd95-4492-4f32-a3b7-ae17cb1bd17e None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:21.250986 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:21.315240 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2371dd95-4492-4f32-a3b7-ae17cb1bd17e None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:21.352747 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39694589-1db6-40d5-aed5-39e6a0ef91ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:21.353388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39694589-1db6-40d5-aed5-39e6a0ef91ac None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:21.353665 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39694589-1db6-40d5-aed5-39e6a0ef91ac None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:21.353939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39694589-1db6-40d5-aed5-39e6a0ef91ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:21.364318 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39694589-1db6-40d5-aed5-39e6a0ef91ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:21.366081 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39694589-1db6-40d5-aed5-39e6a0ef91ac None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:21.367683 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829c7d6-36f6-431c-870f-883b9eb76b27 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:21.368189 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829c7d6-36f6-431c-870f-883b9eb76b27 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:21.368988 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829c7d6-36f6-431c-870f-883b9eb76b27 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:21.370745 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 549/1073] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:08:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:08:21.375913 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c147a-10dd-4ea0-a91f-b95d160562f3 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:21.376829 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c147a-10dd-4ea0-a91f-b95d160562f3 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:21.377222 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c147a-10dd-4ea0-a91f-b95d160562f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:21.382990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-39694589-1db6-40d5-aed5-39e6a0ef91ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:08:21.382990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:21.382990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:21.382990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:21.382990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:21.382990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:21.382990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:21.382990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:21.382990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:21.382990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:21.382990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:21.382990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:21.382990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:08:21.382990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:08:21.382990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:08:21.384048 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:08:21.384048 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:21.384048 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:21.384048 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:21.384048 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:21.384048 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:21.384048 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:21.384048 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:21.384048 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:21.384048 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:21.384048 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:21.384048 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:21.384048 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:21.384048 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:21.384048 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:21.384048 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:21.384048 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:21.389923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:08:21.389923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:08:21.389923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:08:21.389923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:08:21.389923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 525/1074] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:08:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:08:21.391194 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39694589-1db6-40d5-aed5-39e6a0ef91ac None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:21.397768 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-4b3c147a-10dd-4ea0-a91f-b95d160562f3 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:08:21.399084 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-4b3c147a-10dd-4ea0-a91f-b95d160562f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:08:21.476514 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 550/1075] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:08:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 14:08:21.487193 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:21.496425 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39694589-1db6-40d5-aed5-39e6a0ef91ac None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:21.501597 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c147a-10dd-4ea0-a91f-b95d160562f3 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:21.531560 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0456f70a-1fc0-481d-ac29-a5fd50ce8fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:21.532212 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456f70a-1fc0-481d-ac29-a5fd50ce8fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:21.532981 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456f70a-1fc0-481d-ac29-a5fd50ce8fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:21.533355 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456f70a-1fc0-481d-ac29-a5fd50ce8fb3 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:21.534049 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0456f70a-1fc0-481d-ac29-a5fd50ce8fb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:21.534358 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0456f70a-1fc0-481d-ac29-a5fd50ce8fb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:21.535239 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:21.535720 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0456f70a-1fc0-481d-ac29-a5fd50ce8fb3 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:21.547295 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 526/1076] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:08:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:21.552929 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0456f70a-1fc0-481d-ac29-a5fd50ce8fb3 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:21.608664 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c147a-10dd-4ea0-a91f-b95d160562f3 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:21.644105 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b062c-1306-4ad6-bbdf-3006ad81056a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:21.644692 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b062c-1306-4ad6-bbdf-3006ad81056a None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:21.644971 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b062c-1306-4ad6-bbdf-3006ad81056a None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:21.645251 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b062c-1306-4ad6-bbdf-3006ad81056a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:21.653744 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0456f70a-1fc0-481d-ac29-a5fd50ce8fb3 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:21.656969 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b062c-1306-4ad6-bbdf-3006ad81056a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:08:21.661943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-d79b062c-1306-4ad6-bbdf-3006ad81056a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:08:21.661943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:21.661943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:21.661943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:21.661943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:21.661943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:21.661943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:21.661943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:21.661943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:21.661943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:21.661943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:21.661943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:21.661943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:08:21.661943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:08:21.661943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:08:21.661943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:08:21.662916 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:08:21.662916 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:08:21.662916 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:08:21.662916 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:08:21.662916 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:21.662916 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:21.662916 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:21.662916 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:21.662916 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:21.662916 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:21.662916 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:21.662916 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:21.662916 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:21.662916 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:21.662916 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:21.662916 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:21.663756 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:21.663756 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:21.663756 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:21.663756 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:21.663756 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:08:21.663756 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:08:21.663756 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:08:21.663756 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:08:21.663756 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:08:21.663756 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:08:21.663756 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:08:21.663756 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:21.663756 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 551/1077] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Aug 5 14:08:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:08:21.669252 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b062c-1306-4ad6-bbdf-3006ad81056a None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:21.689688 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602e097-ec9f-4601-9031-0d3d53badec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:21.690342 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602e097-ec9f-4601-9031-0d3d53badec5 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:21.690615 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602e097-ec9f-4601-9031-0d3d53badec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:21.690878 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602e097-ec9f-4601-9031-0d3d53badec5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:21.697620 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6602e097-ec9f-4601-9031-0d3d53badec5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:21.699047 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6602e097-ec9f-4601-9031-0d3d53badec5 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:21.710852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-6602e097-ec9f-4601-9031-0d3d53badec5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:08:21.710852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:21.710852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:21.710852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:21.710852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:21.710852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:21.710852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:21.710852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:21.710852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:21.710852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:21.710852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:21.710852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:21.710852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 14:08:21.710852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 14:08:21.710852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 14:08:21.711936 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 14:08:21.711936 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:21.711936 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:21.711936 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:21.711936 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:21.711936 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:21.711936 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:21.711936 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:21.711936 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:21.711936 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:21.711936 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:21.711936 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:21.711936 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:21.711936 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:21.711936 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:21.711936 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:21.711936 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:21.713111 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 14:08:21.713111 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 14:08:21.713111 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:08:21.713111 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:08:21.713111 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:08:21.713111 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:08:21.713111 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:08:21.713111 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:08:21.713111 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 527/1078] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:08:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:08:21.717778 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602e097-ec9f-4601-9031-0d3d53badec5 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:21.730398 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:21.758251 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:21.770057 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b062c-1306-4ad6-bbdf-3006ad81056a None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:21.803398 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2bf2d9-aebc-47ae-81ab-30ee1e13cd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:21.804011 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2bf2d9-aebc-47ae-81ab-30ee1e13cd1d None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:21.804311 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2bf2d9-aebc-47ae-81ab-30ee1e13cd1d None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:21.804626 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2bf2d9-aebc-47ae-81ab-30ee1e13cd1d None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:21.805328 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2bf2d9-aebc-47ae-81ab-30ee1e13cd1d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:21.805627 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2bf2d9-aebc-47ae-81ab-30ee1e13cd1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:21.806975 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2bf2d9-aebc-47ae-81ab-30ee1e13cd1d None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:21.816259 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 552/1079] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:08:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:08:21.822225 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2bf2d9-aebc-47ae-81ab-30ee1e13cd1d None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:21.823061 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602e097-ec9f-4601-9031-0d3d53badec5 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:21.857204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2144daea-06f2-459b-894d-d1ee3a73639b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:21.857815 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144daea-06f2-459b-894d-d1ee3a73639b None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:21.858096 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144daea-06f2-459b-894d-d1ee3a73639b None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:21.858370 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144daea-06f2-459b-894d-d1ee3a73639b None admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:21.859114 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2144daea-06f2-459b-894d-d1ee3a73639b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:21.859420 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2144daea-06f2-459b-894d-d1ee3a73639b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:21.861415 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2144daea-06f2-459b-894d-d1ee3a73639b None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:21.873642 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 528/1080] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Aug 5 14:08:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:08:21.879699 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2144daea-06f2-459b-894d-d1ee3a73639b None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:21.921387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2bf2d9-aebc-47ae-81ab-30ee1e13cd1d None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:21.946851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:21.955453 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926bf299-27f9-4bd5-b95e-0e83a8bf83b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:21.956092 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926bf299-27f9-4bd5-b95e-0e83a8bf83b3 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:21.956369 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926bf299-27f9-4bd5-b95e-0e83a8bf83b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:21.964435 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926bf299-27f9-4bd5-b95e-0e83a8bf83b3 None admin] PATH_INFO: `/v3/users/943fee42863743b5b6f0f327b8754726` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:21.964435 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926bf299-27f9-4bd5-b95e-0e83a8bf83b3 None admin] RBAC: Authorizing `identity:get_user(user_id=943fee42863743b5b6f0f327b8754726)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:21.964435 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926bf299-27f9-4bd5-b95e-0e83a8bf83b3 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:21.964435 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 553/1081] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:08:21 2022] GET /identity/v3/users/943fee42863743b5b6f0f327b8754726 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:08:21.966430 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926bf299-27f9-4bd5-b95e-0e83a8bf83b3 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:21.977688 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2144daea-06f2-459b-894d-d1ee3a73639b None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:21.984703 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:22.014923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f1dea-8e1c-4ad3-ae3e-e35484e0f21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:22.015584 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f1dea-8e1c-4ad3-ae3e-e35484e0f21d None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:22.015861 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f1dea-8e1c-4ad3-ae3e-e35484e0f21d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:22.016131 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f1dea-8e1c-4ad3-ae3e-e35484e0f21d None admin] PATH_INFO: `/v3/projects/319973e011c54b4ca1f2198c8a5e135b/users/5ec4940c99fc4fa8a5bc7cdabc4558fb/roles/5c0fa9dd58a349cba21555779c761f0b` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:22.018366 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f1dea-8e1c-4ad3-ae3e-e35484e0f21d None admin] RBAC: Authorizing `identity:create_grant(project_id=319973e011c54b4ca1f2198c8a5e135b, user_id=5ec4940c99fc4fa8a5bc7cdabc4558fb, role_id=5c0fa9dd58a349cba21555779c761f0b)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:22.019782 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f1dea-8e1c-4ad3-ae3e-e35484e0f21d None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:22.066114 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926bf299-27f9-4bd5-b95e-0e83a8bf83b3 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:22.088310 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 529/1082] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Aug 5 14:08:21 2022] PUT /identity/v3/projects/319973e011c54b4ca1f2198c8a5e135b/users/5ec4940c99fc4fa8a5bc7cdabc4558fb/roles/5c0fa9dd58a349cba21555779c761f0b => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:08:22.102387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032db4c5-123c-4973-96ff-c091745c8a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:22.103057 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032db4c5-123c-4973-96ff-c091745c8a8e None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:22.103334 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032db4c5-123c-4973-96ff-c091745c8a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:22.103608 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032db4c5-123c-4973-96ff-c091745c8a8e None admin] PATH_INFO: `/v3/projects/2a47193e470c44c9988a1094aa4dddc3` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:22.104892 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032db4c5-123c-4973-96ff-c091745c8a8e None admin] RBAC: Authorizing `identity:get_project(project_id=2a47193e470c44c9988a1094aa4dddc3)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:22.106462 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032db4c5-123c-4973-96ff-c091745c8a8e None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:22.108226 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 554/1083] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:08:21 2022] GET /identity/v3/projects/2a47193e470c44c9988a1094aa4dddc3 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:08:22.115224 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f1dea-8e1c-4ad3-ae3e-e35484e0f21d None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:22.221908 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f1dea-8e1c-4ad3-ae3e-e35484e0f21d None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:22.252003 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:22.258135 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e84d6-2d62-42b6-87ab-9196d93df068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:22.258803 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e84d6-2d62-42b6-87ab-9196d93df068 None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:22.259081 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e84d6-2d62-42b6-87ab-9196d93df068 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:22.263552 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e84d6-2d62-42b6-87ab-9196d93df068 None admin] PATH_INFO: `/v3/projects/2a47193e470c44c9988a1094aa4dddc3/users/943fee42863743b5b6f0f327b8754726/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:22.263552 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1e84d6-2d62-42b6-87ab-9196d93df068 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a47193e470c44c9988a1094aa4dddc3, user_id=943fee42863743b5b6f0f327b8754726, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:22.263552 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1e84d6-2d62-42b6-87ab-9196d93df068 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:22.341861 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 530/1084] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Aug 5 14:08:22 2022] PUT /identity/v3/projects/2a47193e470c44c9988a1094aa4dddc3/users/943fee42863743b5b6f0f327b8754726/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:08:24.089980 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05952ace-6ea6-46a3-9038-03f7be5ede62 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:24.090548 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05952ace-6ea6-46a3-9038-03f7be5ede62 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:24.090548 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05952ace-6ea6-46a3-9038-03f7be5ede62 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:24.091850 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 555/1085] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:08:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:08:24.097806 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54438e8b-9545-4cb7-826a-ed2a6350aa4b None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:24.098227 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54438e8b-9545-4cb7-826a-ed2a6350aa4b None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:24.098331 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54438e8b-9545-4cb7-826a-ed2a6350aa4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:24.116000 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-54438e8b-9545-4cb7-826a-ed2a6350aa4b None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:08:24.117270 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-54438e8b-9545-4cb7-826a-ed2a6350aa4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:08:24.180646 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 531/1086] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:08:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:08:24.203343 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05952ace-6ea6-46a3-9038-03f7be5ede62 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:24.287245 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:24.302239 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05952ace-6ea6-46a3-9038-03f7be5ede62 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:24.335531 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac329a3-82fe-49e4-81da-a2f52531cd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:24.336156 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac329a3-82fe-49e4-81da-a2f52531cd1c None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:24.336445 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac329a3-82fe-49e4-81da-a2f52531cd1c None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:24.336715 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac329a3-82fe-49e4-81da-a2f52531cd1c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:24.347134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac329a3-82fe-49e4-81da-a2f52531cd1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:08:24.351914 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-2ac329a3-82fe-49e4-81da-a2f52531cd1c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 14:08:24.351914 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:24.351914 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:24.351914 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:24.351914 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:24.351914 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:24.351914 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:24.351914 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:24.351914 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:24.351914 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:24.351914 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:24.351914 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:24.351914 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:08:24.351914 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:08:24.351914 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:08:24.351914 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:08:24.352922 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:08:24.352922 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:08:24.352922 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:08:24.352922 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:08:24.352922 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:24.352922 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:24.352922 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:24.352922 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:24.352922 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:24.352922 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:24.352922 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:24.352922 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:24.352922 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:24.352922 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:24.352922 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:24.352922 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:24.353767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:24.353767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:24.353767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:24.353767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:24.353767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:08:24.353767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:08:24.353767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:08:24.353767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:08:24.353767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:08:24.353767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:08:24.353767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 14:08:24.353767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:24.353767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 556/1087] 10.222.0.26 () {56 vars in 1120 bytes} [Fri Aug 5 14:08:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:08:24.359042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54438e8b-9545-4cb7-826a-ed2a6350aa4b None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:24.365248 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec7425-133a-4b39-a29d-05c134a3e72c None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:24.365532 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec7425-133a-4b39-a29d-05c134a3e72c None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:24.365805 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec7425-133a-4b39-a29d-05c134a3e72c None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:24.367109 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 557/1088] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:08:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:08:24.372143 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d3630-81e1-4db4-8630-5b9b076bf30c None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:24.372461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d3630-81e1-4db4-8630-5b9b076bf30c None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:24.372745 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d3630-81e1-4db4-8630-5b9b076bf30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:24.391188 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-f85d3630-81e1-4db4-8630-5b9b076bf30c None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:08:24.393200 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-f85d3630-81e1-4db4-8630-5b9b076bf30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:08:24.458402 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54438e8b-9545-4cb7-826a-ed2a6350aa4b None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:24.460883 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 558/1089] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:08:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 14:08:24.479061 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:24.483196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d3630-81e1-4db4-8630-5b9b076bf30c None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:24.492808 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d4b978-2180-4255-aec6-103aabfc93e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:24.493428 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4b978-2180-4255-aec6-103aabfc93e2 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:24.497336 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4b978-2180-4255-aec6-103aabfc93e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:24.497336 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4b978-2180-4255-aec6-103aabfc93e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:24.497336 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d4b978-2180-4255-aec6-103aabfc93e2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:24.497336 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d4b978-2180-4255-aec6-103aabfc93e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:24.497336 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d4b978-2180-4255-aec6-103aabfc93e2 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:24.505108 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 532/1090] 10.222.0.26 () {56 vars in 1115 bytes} [Fri Aug 5 14:08:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:24.510684 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d4b978-2180-4255-aec6-103aabfc93e2 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:24.561810 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:24.585662 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d3630-81e1-4db4-8630-5b9b076bf30c None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:24.612403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d4b978-2180-4255-aec6-103aabfc93e2 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:24.620764 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d86c80-3830-4eb2-92e4-ce18808a5d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:24.621404 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d86c80-3830-4eb2-92e4-ce18808a5d61 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:24.621679 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d86c80-3830-4eb2-92e4-ce18808a5d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:24.621990 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d86c80-3830-4eb2-92e4-ce18808a5d61 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:24.632494 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d86c80-3830-4eb2-92e4-ce18808a5d61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:08:24.637449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-b8d86c80-3830-4eb2-92e4-ce18808a5d61 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:08:24.637449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:24.637449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:24.637449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:24.637449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:24.637449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:24.637449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:24.637449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:24.637449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:24.637449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:24.637449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:24.637449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:24.637449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:08:24.637449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:08:24.637449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:08:24.637449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:08:24.641093 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:08:24.641093 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:08:24.641093 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:08:24.641093 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:08:24.641093 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:24.641093 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:24.641093 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:24.641093 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:24.641093 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:24.641093 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:24.641093 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:24.641093 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:24.641093 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:24.641093 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:24.641093 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:24.641093 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:24.642682 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:24.642682 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:24.642682 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:24.642682 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:24.642682 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:08:24.642682 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:08:24.642682 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:08:24.642682 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:08:24.642682 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:08:24.642682 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:08:24.642682 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:08:24.642682 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:24.642682 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 559/1091] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Aug 5 14:08:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:08:24.645648 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d86c80-3830-4eb2-92e4-ce18808a5d61 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:24.654313 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b676abf-57f5-4f5e-8bc7-9fc6e316165a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:24.654961 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b676abf-57f5-4f5e-8bc7-9fc6e316165a None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:24.655239 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b676abf-57f5-4f5e-8bc7-9fc6e316165a None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:24.655509 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b676abf-57f5-4f5e-8bc7-9fc6e316165a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:24.666386 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b676abf-57f5-4f5e-8bc7-9fc6e316165a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:24.667891 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b676abf-57f5-4f5e-8bc7-9fc6e316165a None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:24.681750 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-6b676abf-57f5-4f5e-8bc7-9fc6e316165a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:08:24.681750 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:24.681750 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:24.681750 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:24.681750 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:24.681750 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:24.681750 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:24.681750 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:24.681750 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:24.681750 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:24.681750 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:24.681750 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:24.681750 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:08:24.681750 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:08:24.681750 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:08:24.682691 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:08:24.682691 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:24.682691 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:24.682691 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:24.682691 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:24.682691 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:24.682691 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:24.682691 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:24.682691 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:24.682691 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:24.682691 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:24.682691 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:24.682691 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:24.682691 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:24.682691 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:24.682691 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:24.682691 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:24.683946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:08:24.683946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:08:24.683946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:08:24.683946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:08:24.683946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 533/1092] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:08:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:08:24.689080 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b676abf-57f5-4f5e-8bc7-9fc6e316165a None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:24.723301 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:24.748978 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d86c80-3830-4eb2-92e4-ce18808a5d61 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:24.784224 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec908c1b-cea0-4218-bfec-2fe28d4ef231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:24.784968 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec908c1b-cea0-4218-bfec-2fe28d4ef231 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:24.785246 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec908c1b-cea0-4218-bfec-2fe28d4ef231 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:24.785517 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec908c1b-cea0-4218-bfec-2fe28d4ef231 None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:24.786219 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec908c1b-cea0-4218-bfec-2fe28d4ef231 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:24.786520 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec908c1b-cea0-4218-bfec-2fe28d4ef231 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:24.787868 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec908c1b-cea0-4218-bfec-2fe28d4ef231 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:24.788197 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b676abf-57f5-4f5e-8bc7-9fc6e316165a None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:24.790060 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:24.796829 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 560/1093] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:08:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:08:24.803249 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec908c1b-cea0-4218-bfec-2fe28d4ef231 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:24.824334 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac5276f-0cfc-438f-bb2a-be62dd28ac64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:24.824982 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5276f-0cfc-438f-bb2a-be62dd28ac64 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:24.825266 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5276f-0cfc-438f-bb2a-be62dd28ac64 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:24.825535 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5276f-0cfc-438f-bb2a-be62dd28ac64 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:24.826217 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac5276f-0cfc-438f-bb2a-be62dd28ac64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:24.826519 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac5276f-0cfc-438f-bb2a-be62dd28ac64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:24.827864 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac5276f-0cfc-438f-bb2a-be62dd28ac64 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:24.844168 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 534/1094] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:08:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:08:24.846370 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac5276f-0cfc-438f-bb2a-be62dd28ac64 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:24.904660 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec908c1b-cea0-4218-bfec-2fe28d4ef231 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:24.938676 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccadb2b5-828d-4e41-b2d4-17a566fae752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:24.939301 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadb2b5-828d-4e41-b2d4-17a566fae752 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:24.939576 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadb2b5-828d-4e41-b2d4-17a566fae752 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:24.939849 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadb2b5-828d-4e41-b2d4-17a566fae752 None admin] PATH_INFO: `/v3/users/943fee42863743b5b6f0f327b8754726` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:24.940036 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:24.941391 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccadb2b5-828d-4e41-b2d4-17a566fae752 None admin] RBAC: Authorizing `identity:get_user(user_id=943fee42863743b5b6f0f327b8754726)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:24.942811 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccadb2b5-828d-4e41-b2d4-17a566fae752 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:24.944405 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 561/1095] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:08:24 2022] GET /identity/v3/users/943fee42863743b5b6f0f327b8754726 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:08:24.946736 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac5276f-0cfc-438f-bb2a-be62dd28ac64 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:24.949993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccadb2b5-828d-4e41-b2d4-17a566fae752 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:24.982679 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b72bb9-6432-4e3b-9b04-dba8de7e60ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:24.983400 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b72bb9-6432-4e3b-9b04-dba8de7e60ad None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:24.983741 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b72bb9-6432-4e3b-9b04-dba8de7e60ad None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:24.984053 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b72bb9-6432-4e3b-9b04-dba8de7e60ad None admin] PATH_INFO: `/v3/users/nova` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:24.991774 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b72bb9-6432-4e3b-9b04-dba8de7e60ad None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:24.993234 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b72bb9-6432-4e3b-9b04-dba8de7e60ad None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:25.005202 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-15b72bb9-6432-4e3b-9b04-dba8de7e60ad None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 14:08:25.005202 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:25.005202 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:25.005202 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:25.005202 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:25.005202 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:25.005202 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:25.005202 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:25.005202 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:25.005202 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:25.005202 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:25.005202 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:25.005202 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 14:08:25.005202 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 14:08:25.005202 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 14:08:25.005202 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 14:08:25.006203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:25.006203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:25.006203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:08:25.006203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:08:25.006203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:08:25.006203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:08:25.006203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:25.006203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:25.006203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:25.006203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:25.006203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:25.006203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:25.006203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:25.006203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:25.006203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:25.006203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:25.006203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:25.006203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:25.007724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:25.007724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:25.007724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 14:08:25.007724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 14:08:25.007724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 14:08:25.007724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 14:08:25.007724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 14:08:25.007724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 14:08:25.007724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 14:08:25.007724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:08:25.007724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 535/1096] 10.222.0.26 () {56 vars in 1093 bytes} [Fri Aug 5 14:08:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:08:25.011901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b72bb9-6432-4e3b-9b04-dba8de7e60ad None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:25.015203 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:25.051107 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccadb2b5-828d-4e41-b2d4-17a566fae752 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:25.084547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1433e778-b1d6-4449-ac23-a28f427b394a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:25.085164 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1433e778-b1d6-4449-ac23-a28f427b394a None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:25.085442 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1433e778-b1d6-4449-ac23-a28f427b394a None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:25.085714 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1433e778-b1d6-4449-ac23-a28f427b394a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:25.112768 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b72bb9-6432-4e3b-9b04-dba8de7e60ad None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:25.153328 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b779ed-ea1f-402a-a235-a5a8f115c499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:25.153941 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b779ed-ea1f-402a-a235-a5a8f115c499 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:25.154222 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b779ed-ea1f-402a-a235-a5a8f115c499 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:25.154498 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b779ed-ea1f-402a-a235-a5a8f115c499 None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:25.154985 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1433e778-b1d6-4449-ac23-a28f427b394a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:25.155412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b779ed-ea1f-402a-a235-a5a8f115c499 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:25.155577 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b779ed-ea1f-402a-a235-a5a8f115c499 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:25.156389 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1433e778-b1d6-4449-ac23-a28f427b394a None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:25.157052 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b779ed-ea1f-402a-a235-a5a8f115c499 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:25.173946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 536/1097] 10.222.0.26 () {56 vars in 1133 bytes} [Fri Aug 5 14:08:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:08:25.179613 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b779ed-ea1f-402a-a235-a5a8f115c499 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:25.193861 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:25.221190 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:25.255456 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 562/1098] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:08:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 05 14:08:25.261479 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1433e778-b1d6-4449-ac23-a28f427b394a None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:25.277098 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b779ed-ea1f-402a-a235-a5a8f115c499 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:25.309563 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805484a9-2e40-45a2-b876-7623e02373c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:25.310171 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805484a9-2e40-45a2-b876-7623e02373c1 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:25.310444 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805484a9-2e40-45a2-b876-7623e02373c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:25.310714 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805484a9-2e40-45a2-b876-7623e02373c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:25.357970 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1433e778-b1d6-4449-ac23-a28f427b394a None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:25.376443 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805484a9-2e40-45a2-b876-7623e02373c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:25.377851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805484a9-2e40-45a2-b876-7623e02373c1 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:25.390864 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:25.392039 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd0527a-23d0-4b32-9b78-756a7dbf9508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:25.392992 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0527a-23d0-4b32-9b78-756a7dbf9508 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:25.393267 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0527a-23d0-4b32-9b78-756a7dbf9508 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:25.393541 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0527a-23d0-4b32-9b78-756a7dbf9508 None admin] PATH_INFO: `/v3/projects/2a47193e470c44c9988a1094aa4dddc3` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:25.394697 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd0527a-23d0-4b32-9b78-756a7dbf9508 None admin] RBAC: Authorizing `identity:get_project(project_id=2a47193e470c44c9988a1094aa4dddc3)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:25.396100 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd0527a-23d0-4b32-9b78-756a7dbf9508 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:25.397821 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 563/1099] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:08:25 2022] GET /identity/v3/projects/2a47193e470c44c9988a1094aa4dddc3 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:08:25.403731 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd0527a-23d0-4b32-9b78-756a7dbf9508 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:25.436873 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:25.476231 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 537/1100] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:08:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 05 14:08:25.481942 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805484a9-2e40-45a2-b876-7623e02373c1 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:25.509541 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd0527a-23d0-4b32-9b78-756a7dbf9508 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:25.547343 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa2bb9a-1319-41ac-afc0-3388556a4193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:25.548708 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2bb9a-1319-41ac-afc0-3388556a4193 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:25.548983 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2bb9a-1319-41ac-afc0-3388556a4193 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:25.549255 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2bb9a-1319-41ac-afc0-3388556a4193 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:25.550040 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa2bb9a-1319-41ac-afc0-3388556a4193 None admin] RBAC: Adding query filter params (role.id=299c794c7acb4a5ca36288c3191aad10, scope.project.id=2a47193e470c44c9988a1094aa4dddc3, user.id=943fee42863743b5b6f0f327b8754726) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:25.550349 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa2bb9a-1319-41ac-afc0-3388556a4193 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:25.551772 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa2bb9a-1319-41ac-afc0-3388556a4193 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:25.557507 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 564/1101] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:08:25 2022] GET /identity/v3/role_assignments?user.id=943fee42863743b5b6f0f327b8754726&scope.project.id=2a47193e470c44c9988a1094aa4dddc3&role.id=299c794c7acb4a5ca36288c3191aad10 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:08:25.582285 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805484a9-2e40-45a2-b876-7623e02373c1 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:25.596700 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:25.619447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d3a07-69ff-4698-afd9-539038bcc706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:25.620094 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d3a07-69ff-4698-afd9-539038bcc706 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:25.620428 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d3a07-69ff-4698-afd9-539038bcc706 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:25.620695 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d3a07-69ff-4698-afd9-539038bcc706 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:25.633415 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52d3a07-69ff-4698-afd9-539038bcc706 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:25.634994 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52d3a07-69ff-4698-afd9-539038bcc706 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:25.650871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-d52d3a07-69ff-4698-afd9-539038bcc706 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:08:25.650871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:25.650871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:25.650871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:25.650871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:25.650871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:25.650871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:25.650871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:25.650871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:25.650871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:25.650871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:25.650871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:25.650871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:08:25.650871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:08:25.650871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:08:25.651907 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:08:25.651907 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:25.651907 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:25.651907 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:25.651907 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:25.651907 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:25.651907 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:25.651907 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:25.651907 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:25.651907 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:25.651907 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:25.651907 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:25.651907 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:25.651907 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:25.651907 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:25.651907 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:25.651907 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:25.652887 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:08:25.652887 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:08:25.652887 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:08:25.652887 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:08:25.652887 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 538/1102] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:08:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:08:25.658591 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa2bb9a-1319-41ac-afc0-3388556a4193 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:25.762786 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:25.766669 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa2bb9a-1319-41ac-afc0-3388556a4193 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:25.799961 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096242d7-48d3-45da-8968-a0a93472df76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:25.800610 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096242d7-48d3-45da-8968-a0a93472df76 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:25.800894 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096242d7-48d3-45da-8968-a0a93472df76 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:25.801167 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096242d7-48d3-45da-8968-a0a93472df76 None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:25.801854 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096242d7-48d3-45da-8968-a0a93472df76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:25.802154 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096242d7-48d3-45da-8968-a0a93472df76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:25.803505 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096242d7-48d3-45da-8968-a0a93472df76 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:25.815977 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 565/1103] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:08:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:25.821650 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d3a07-69ff-4698-afd9-539038bcc706 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:25.918244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d3a07-69ff-4698-afd9-539038bcc706 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:25.951834 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4355796-4742-400b-8c5b-1511350b0a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:25.952560 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4355796-4742-400b-8c5b-1511350b0a94 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:25.952841 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4355796-4742-400b-8c5b-1511350b0a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:25.953119 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4355796-4742-400b-8c5b-1511350b0a94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:25.959898 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4355796-4742-400b-8c5b-1511350b0a94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:25.961354 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4355796-4742-400b-8c5b-1511350b0a94 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:25.973939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-a4355796-4742-400b-8c5b-1511350b0a94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:08:25.973939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:25.973939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:25.973939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:25.973939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:25.973939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:25.973939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:25.973939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:25.973939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:25.973939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:25.973939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:25.973939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:25.973939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 14:08:25.973939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 14:08:25.973939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 14:08:25.974849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 14:08:25.974849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:25.974849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:25.974849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:25.974849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:25.974849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:25.974849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:25.974849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:25.974849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:25.974849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:25.974849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:25.974849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:25.974849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:25.974849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:25.974849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:25.974849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:25.974849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:25.975763 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 14:08:25.975763 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 14:08:25.975763 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:08:25.975763 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:08:25.975763 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:08:25.975763 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:08:25.975763 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:08:25.975763 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:08:25.975763 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 539/1104] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:08:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:08:25.980971 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096242d7-48d3-45da-8968-a0a93472df76 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:26.078321 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096242d7-48d3-45da-8968-a0a93472df76 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:26.111731 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f9a8be-7774-4f63-8dff-bd699a1a5b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:26.112343 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f9a8be-7774-4f63-8dff-bd699a1a5b3e None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:26.112646 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f9a8be-7774-4f63-8dff-bd699a1a5b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:26.112919 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f9a8be-7774-4f63-8dff-bd699a1a5b3e None admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:26.113659 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f9a8be-7774-4f63-8dff-bd699a1a5b3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:26.113965 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f9a8be-7774-4f63-8dff-bd699a1a5b3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:26.115371 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f9a8be-7774-4f63-8dff-bd699a1a5b3e None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:26.126840 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 566/1105] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Aug 5 14:08:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:26.133189 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4355796-4742-400b-8c5b-1511350b0a94 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:26.158810 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:26.229905 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4355796-4742-400b-8c5b-1511350b0a94 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:26.263222 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66fe60-f81f-4452-8bfd-f80dbe049e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:26.263824 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66fe60-f81f-4452-8bfd-f80dbe049e97 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:26.264096 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66fe60-f81f-4452-8bfd-f80dbe049e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:26.264361 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66fe60-f81f-4452-8bfd-f80dbe049e97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:26.265237 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad66fe60-f81f-4452-8bfd-f80dbe049e97 None admin] RBAC: Adding query filter params (role.id=5c0fa9dd58a349cba21555779c761f0b, scope.project.id=319973e011c54b4ca1f2198c8a5e135b, user.id=5ec4940c99fc4fa8a5bc7cdabc4558fb) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:26.265538 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad66fe60-f81f-4452-8bfd-f80dbe049e97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:26.267490 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad66fe60-f81f-4452-8bfd-f80dbe049e97 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:26.273377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 540/1106] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:08:26 2022] GET /identity/v3/role_assignments?user.id=5ec4940c99fc4fa8a5bc7cdabc4558fb&scope.project.id=319973e011c54b4ca1f2198c8a5e135b&role.id=5c0fa9dd58a349cba21555779c761f0b => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:27.577858 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7388f-b445-4718-b72c-82cd44fdf736 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:27.578362 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7388f-b445-4718-b72c-82cd44fdf736 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:27.578456 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7388f-b445-4718-b72c-82cd44fdf736 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:27.580436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 567/1107] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:08:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:08:27.585416 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c039df6b-5d0f-425a-a87e-6b9e2d451eb1 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:27.590614 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c039df6b-5d0f-425a-a87e-6b9e2d451eb1 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:27.590614 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c039df6b-5d0f-425a-a87e-6b9e2d451eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:27.603697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-c039df6b-5d0f-425a-a87e-6b9e2d451eb1 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:08:27.604918 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c039df6b-5d0f-425a-a87e-6b9e2d451eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:08:27.668968 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 541/1108] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:08:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:08:27.693184 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7388f-b445-4718-b72c-82cd44fdf736 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:27.714465 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:27.794391 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7388f-b445-4718-b72c-82cd44fdf736 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:27.827053 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d8ca2-dfbe-41e6-a692-45f8c22bdd1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:27.827682 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d8ca2-dfbe-41e6-a692-45f8c22bdd1e None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:27.827956 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d8ca2-dfbe-41e6-a692-45f8c22bdd1e None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:27.828222 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d8ca2-dfbe-41e6-a692-45f8c22bdd1e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:27.829639 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968d8ca2-dfbe-41e6-a692-45f8c22bdd1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:27.831058 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968d8ca2-dfbe-41e6-a692-45f8c22bdd1e None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:27.832652 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 568/1109] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:08:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:27.838334 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c039df6b-5d0f-425a-a87e-6b9e2d451eb1 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:27.855874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:27.933815 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c039df6b-5d0f-425a-a87e-6b9e2d451eb1 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:27.965875 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ffe34-e8b8-41a2-b815-6e3f4e2919a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:27.966470 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ffe34-e8b8-41a2-b815-6e3f4e2919a3 None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:27.966740 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ffe34-e8b8-41a2-b815-6e3f4e2919a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:27.967003 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ffe34-e8b8-41a2-b815-6e3f4e2919a3 None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:27.967660 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ffe34-e8b8-41a2-b815-6e3f4e2919a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:27.969095 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ffe34-e8b8-41a2-b815-6e3f4e2919a3 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:28.055582 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 542/1110] 10.222.0.26 () {60 vars in 1132 bytes} [Fri Aug 5 14:08:27 2022] POST /identity/v3/users => generated 286 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:08:30.055770 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d6b99c-f479-4f72-99c5-f6789913f679 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:30.056303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d6b99c-f479-4f72-99c5-f6789913f679 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:30.056303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d6b99c-f479-4f72-99c5-f6789913f679 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:30.057725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 569/1111] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:08:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:08:30.062867 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aabee4-aa4a-481b-8c29-6cfc843e9db3 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:30.063262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aabee4-aa4a-481b-8c29-6cfc843e9db3 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:30.063419 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aabee4-aa4a-481b-8c29-6cfc843e9db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:30.082027 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-27aabee4-aa4a-481b-8c29-6cfc843e9db3 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:08:30.083192 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-27aabee4-aa4a-481b-8c29-6cfc843e9db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:08:30.147578 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 543/1112] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:08:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:08:30.170258 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d6b99c-f479-4f72-99c5-f6789913f679 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:30.255833 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:30.270900 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d6b99c-f479-4f72-99c5-f6789913f679 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:30.303835 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19afa18-9d9f-46ed-80a2-7aaa6ae5c6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:30.304485 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19afa18-9d9f-46ed-80a2-7aaa6ae5c6bf None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:30.304767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19afa18-9d9f-46ed-80a2-7aaa6ae5c6bf None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:30.305040 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19afa18-9d9f-46ed-80a2-7aaa6ae5c6bf None admin] PATH_INFO: `/v3/roles/199fd3093a824ba585b1135c8bf363be` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:30.306536 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19afa18-9d9f-46ed-80a2-7aaa6ae5c6bf None admin] RBAC: Authorizing `identity:get_role(role_id=199fd3093a824ba585b1135c8bf363be)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:30.307834 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19afa18-9d9f-46ed-80a2-7aaa6ae5c6bf None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:30.309179 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 570/1113] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:08:30 2022] GET /identity/v3/roles/199fd3093a824ba585b1135c8bf363be => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:30.315907 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aabee4-aa4a-481b-8c29-6cfc843e9db3 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:30.318954 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:30.412092 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aabee4-aa4a-481b-8c29-6cfc843e9db3 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:30.445235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548d157-8dc9-45a5-9b28-ac654b87e6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:30.445853 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548d157-8dc9-45a5-9b28-ac654b87e6f6 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:30.446124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548d157-8dc9-45a5-9b28-ac654b87e6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:30.446385 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548d157-8dc9-45a5-9b28-ac654b87e6f6 None admin] PATH_INFO: `/v3/users/f08d8490264e411ea5e60490b33d09c5` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:30.459037 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e548d157-8dc9-45a5-9b28-ac654b87e6f6 None admin] RBAC: Authorizing `identity:get_user(user_id=f08d8490264e411ea5e60490b33d09c5)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:30.460470 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e548d157-8dc9-45a5-9b28-ac654b87e6f6 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:30.462041 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 544/1114] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:08:30 2022] GET /identity/v3/users/f08d8490264e411ea5e60490b33d09c5 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:30.467726 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19afa18-9d9f-46ed-80a2-7aaa6ae5c6bf None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:30.563838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19afa18-9d9f-46ed-80a2-7aaa6ae5c6bf None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:30.596217 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0376a1e-1799-4361-a6a3-263145aaf645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:30.596845 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0376a1e-1799-4361-a6a3-263145aaf645 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:30.597120 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0376a1e-1799-4361-a6a3-263145aaf645 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:30.597387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0376a1e-1799-4361-a6a3-263145aaf645 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:30.621001 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:30.663365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0376a1e-1799-4361-a6a3-263145aaf645 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:30.664843 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0376a1e-1799-4361-a6a3-263145aaf645 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:30.815506 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 571/1115] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:08:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 14:08:30.821496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548d157-8dc9-45a5-9b28-ac654b87e6f6 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:30.881329 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:30.919185 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548d157-8dc9-45a5-9b28-ac654b87e6f6 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:30.950844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8146133-eed6-422e-ba48-fe9b5ee702b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:30.951471 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8146133-eed6-422e-ba48-fe9b5ee702b4 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:30.951750 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8146133-eed6-422e-ba48-fe9b5ee702b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:30.952023 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8146133-eed6-422e-ba48-fe9b5ee702b4 None admin] PATH_INFO: `/v3/projects/2a47193e470c44c9988a1094aa4dddc3` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:30.953190 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8146133-eed6-422e-ba48-fe9b5ee702b4 None admin] RBAC: Authorizing `identity:get_project(project_id=2a47193e470c44c9988a1094aa4dddc3)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:30.955182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8146133-eed6-422e-ba48-fe9b5ee702b4 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:30.956844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 545/1116] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:08:30 2022] GET /identity/v3/projects/2a47193e470c44c9988a1094aa4dddc3 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:30.962949 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0376a1e-1799-4361-a6a3-263145aaf645 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:31.017919 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:31.060238 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0376a1e-1799-4361-a6a3-263145aaf645 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:31.093169 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847d636-aaae-45d6-9c9f-a0159bde28ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:31.093716 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847d636-aaae-45d6-9c9f-a0159bde28ea None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:31.093991 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847d636-aaae-45d6-9c9f-a0159bde28ea None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:31.094962 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847d636-aaae-45d6-9c9f-a0159bde28ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:31.095747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c847d636-aaae-45d6-9c9f-a0159bde28ea None admin] RBAC: Adding query filter params (role.id=199fd3093a824ba585b1135c8bf363be, scope.project.id=2a47193e470c44c9988a1094aa4dddc3, user.id=f08d8490264e411ea5e60490b33d09c5) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:31.096056 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c847d636-aaae-45d6-9c9f-a0159bde28ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:31.097483 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c847d636-aaae-45d6-9c9f-a0159bde28ea None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:31.102683 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 572/1117] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:08:30 2022] GET /identity/v3/role_assignments?user.id=f08d8490264e411ea5e60490b33d09c5&scope.project.id=2a47193e470c44c9988a1094aa4dddc3&role.id=199fd3093a824ba585b1135c8bf363be => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:33.056994 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04c89f-cc7d-4d2d-ad2e-41e39cf7289d None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:33.057527 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04c89f-cc7d-4d2d-ad2e-41e39cf7289d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:33.057527 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04c89f-cc7d-4d2d-ad2e-41e39cf7289d None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:33.058853 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 546/1118] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:08:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:08:33.064667 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975820eb-535c-457c-bbf6-eeafecacbc75 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:33.065048 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975820eb-535c-457c-bbf6-eeafecacbc75 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:33.065210 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975820eb-535c-457c-bbf6-eeafecacbc75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:33.083117 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-975820eb-535c-457c-bbf6-eeafecacbc75 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:08:33.084302 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-975820eb-535c-457c-bbf6-eeafecacbc75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:08:33.149976 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 573/1119] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:08:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:08:33.173172 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04c89f-cc7d-4d2d-ad2e-41e39cf7289d None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:33.271432 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04c89f-cc7d-4d2d-ad2e-41e39cf7289d None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:33.298981 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:33.304166 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0f49c0-3e83-430e-9955-46073e541706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:33.304810 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f49c0-3e83-430e-9955-46073e541706 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:33.305083 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f49c0-3e83-430e-9955-46073e541706 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:33.305350 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f49c0-3e83-430e-9955-46073e541706 None admin] PATH_INFO: `/v3/roles/199fd3093a824ba585b1135c8bf363be` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:33.306762 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0f49c0-3e83-430e-9955-46073e541706 None admin] RBAC: Authorizing `identity:get_role(role_id=199fd3093a824ba585b1135c8bf363be)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:33.308146 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0f49c0-3e83-430e-9955-46073e541706 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:33.309420 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 547/1120] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:08:33 2022] GET /identity/v3/roles/199fd3093a824ba585b1135c8bf363be => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:33.315773 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975820eb-535c-457c-bbf6-eeafecacbc75 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:33.362955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:33.412321 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975820eb-535c-457c-bbf6-eeafecacbc75 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:33.445580 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b79e0c-7a6b-4621-b1c1-e713ea14f7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:33.446201 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b79e0c-7a6b-4621-b1c1-e713ea14f7aa None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:33.446477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b79e0c-7a6b-4621-b1c1-e713ea14f7aa None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:33.446747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b79e0c-7a6b-4621-b1c1-e713ea14f7aa None admin] PATH_INFO: `/v3/users/f08d8490264e411ea5e60490b33d09c5` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:33.447895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b79e0c-7a6b-4621-b1c1-e713ea14f7aa None admin] RBAC: Authorizing `identity:get_user(user_id=f08d8490264e411ea5e60490b33d09c5)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:33.449330 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b79e0c-7a6b-4621-b1c1-e713ea14f7aa None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:33.450905 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 574/1121] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:08:33 2022] GET /identity/v3/users/f08d8490264e411ea5e60490b33d09c5 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:33.457192 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0f49c0-3e83-430e-9955-46073e541706 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:33.554138 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0f49c0-3e83-430e-9955-46073e541706 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:33.592048 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23092acf-c6c0-4896-89ea-4f0c435b7bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:33.592631 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23092acf-c6c0-4896-89ea-4f0c435b7bdc None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:33.592904 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23092acf-c6c0-4896-89ea-4f0c435b7bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:33.593170 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23092acf-c6c0-4896-89ea-4f0c435b7bdc None admin] PATH_INFO: `/v3/projects/2a47193e470c44c9988a1094aa4dddc3` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:33.594491 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23092acf-c6c0-4896-89ea-4f0c435b7bdc None admin] RBAC: Authorizing `identity:get_project(project_id=2a47193e470c44c9988a1094aa4dddc3)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:33.596018 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23092acf-c6c0-4896-89ea-4f0c435b7bdc None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:33.597676 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 548/1122] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:08:33 2022] GET /identity/v3/projects/2a47193e470c44c9988a1094aa4dddc3 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:33.603606 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b79e0c-7a6b-4621-b1c1-e713ea14f7aa None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:33.699260 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b79e0c-7a6b-4621-b1c1-e713ea14f7aa None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:33.715412 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:33.731991 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07aec3f-9965-4aff-8f5e-613707ed32e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:33.732667 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07aec3f-9965-4aff-8f5e-613707ed32e1 None admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:33.732945 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07aec3f-9965-4aff-8f5e-613707ed32e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:33.733216 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07aec3f-9965-4aff-8f5e-613707ed32e1 None admin] PATH_INFO: `/v3/projects/2a47193e470c44c9988a1094aa4dddc3/users/f08d8490264e411ea5e60490b33d09c5/roles/199fd3093a824ba585b1135c8bf363be` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:33.735918 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07aec3f-9965-4aff-8f5e-613707ed32e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a47193e470c44c9988a1094aa4dddc3, user_id=f08d8490264e411ea5e60490b33d09c5, role_id=199fd3093a824ba585b1135c8bf363be)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:33.737356 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07aec3f-9965-4aff-8f5e-613707ed32e1 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:33.824372 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 575/1123] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Aug 5 14:08:33 2022] PUT /identity/v3/projects/2a47193e470c44c9988a1094aa4dddc3/users/f08d8490264e411ea5e60490b33d09c5/roles/199fd3093a824ba585b1135c8bf363be => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:08:35.794961 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706c0d8-34cd-48e3-8694-8a17317c0f91 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:35.795511 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706c0d8-34cd-48e3-8694-8a17317c0f91 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:35.795511 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706c0d8-34cd-48e3-8694-8a17317c0f91 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:35.796764 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 549/1124] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:08:35.802387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7d19b-7880-4d96-8eb8-1bfa7663fe6f None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:35.802821 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7d19b-7880-4d96-8eb8-1bfa7663fe6f None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:35.802923 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7d19b-7880-4d96-8eb8-1bfa7663fe6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:35.820604 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-59f7d19b-7880-4d96-8eb8-1bfa7663fe6f None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:08:35.821777 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-59f7d19b-7880-4d96-8eb8-1bfa7663fe6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:08:35.885525 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 576/1125] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:08:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:08:35.908659 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7706c0d8-34cd-48e3-8694-8a17317c0f91 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:35.973479 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:36.007851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7706c0d8-34cd-48e3-8694-8a17317c0f91 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:36.040927 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d36f70-41ad-4dd0-afcd-f33cdbb10929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:36.041549 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d36f70-41ad-4dd0-afcd-f33cdbb10929 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:36.041825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d36f70-41ad-4dd0-afcd-f33cdbb10929 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:36.042116 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d36f70-41ad-4dd0-afcd-f33cdbb10929 None admin] PATH_INFO: `/v3/roles/199fd3093a824ba585b1135c8bf363be` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:36.043540 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d36f70-41ad-4dd0-afcd-f33cdbb10929 None admin] RBAC: Authorizing `identity:get_role(role_id=199fd3093a824ba585b1135c8bf363be)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:36.044935 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d36f70-41ad-4dd0-afcd-f33cdbb10929 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:36.046308 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 550/1126] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:08:35 2022] GET /identity/v3/roles/199fd3093a824ba585b1135c8bf363be => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:36.052236 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7d19b-7880-4d96-8eb8-1bfa7663fe6f None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:36.147038 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7d19b-7880-4d96-8eb8-1bfa7663fe6f None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:36.154052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:36.180109 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841bd285-fe41-4b94-a5ec-fffbdeaaeda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:36.180770 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841bd285-fe41-4b94-a5ec-fffbdeaaeda1 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:36.181049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841bd285-fe41-4b94-a5ec-fffbdeaaeda1 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:36.181320 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841bd285-fe41-4b94-a5ec-fffbdeaaeda1 None admin] PATH_INFO: `/v3/users/f08d8490264e411ea5e60490b33d09c5` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:36.182471 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841bd285-fe41-4b94-a5ec-fffbdeaaeda1 None admin] RBAC: Authorizing `identity:get_user(user_id=f08d8490264e411ea5e60490b33d09c5)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:36.183882 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841bd285-fe41-4b94-a5ec-fffbdeaaeda1 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:36.185490 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 577/1127] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:08:36 2022] GET /identity/v3/users/f08d8490264e411ea5e60490b33d09c5 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:36.191284 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d36f70-41ad-4dd0-afcd-f33cdbb10929 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:36.289202 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d36f70-41ad-4dd0-afcd-f33cdbb10929 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:36.320949 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbeae3a-e3cc-4f80-ae3a-cf66c74f43ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:36.321489 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbeae3a-e3cc-4f80-ae3a-cf66c74f43ab None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:36.321766 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbeae3a-e3cc-4f80-ae3a-cf66c74f43ab None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:36.322036 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbeae3a-e3cc-4f80-ae3a-cf66c74f43ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:36.324564 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:36.387353 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbeae3a-e3cc-4f80-ae3a-cf66c74f43ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:36.388802 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbeae3a-e3cc-4f80-ae3a-cf66c74f43ab None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:36.482132 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 551/1128] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:08:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 14:08:36.488187 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841bd285-fe41-4b94-a5ec-fffbdeaaeda1 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:36.584287 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841bd285-fe41-4b94-a5ec-fffbdeaaeda1 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:36.616439 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84912a71-b330-4642-bd2c-b3b7afc1d69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:36.617081 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84912a71-b330-4642-bd2c-b3b7afc1d69b None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:36.617358 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84912a71-b330-4642-bd2c-b3b7afc1d69b None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:36.617628 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84912a71-b330-4642-bd2c-b3b7afc1d69b None admin] PATH_INFO: `/v3/projects/2a47193e470c44c9988a1094aa4dddc3` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:36.619029 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84912a71-b330-4642-bd2c-b3b7afc1d69b None admin] RBAC: Authorizing `identity:get_project(project_id=2a47193e470c44c9988a1094aa4dddc3)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:36.620468 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84912a71-b330-4642-bd2c-b3b7afc1d69b None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:36.622104 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 578/1129] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:08:36 2022] GET /identity/v3/projects/2a47193e470c44c9988a1094aa4dddc3 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:36.628827 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbeae3a-e3cc-4f80-ae3a-cf66c74f43ab None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:36.661634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:36.725591 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbeae3a-e3cc-4f80-ae3a-cf66c74f43ab None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:36.759670 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39aa6f4-c325-4829-9240-21eea4fda5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:36.760273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39aa6f4-c325-4829-9240-21eea4fda5fb None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:36.760580 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39aa6f4-c325-4829-9240-21eea4fda5fb None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:36.760852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39aa6f4-c325-4829-9240-21eea4fda5fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:36.761636 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39aa6f4-c325-4829-9240-21eea4fda5fb None admin] RBAC: Adding query filter params (role.id=199fd3093a824ba585b1135c8bf363be, scope.project.id=2a47193e470c44c9988a1094aa4dddc3, user.id=f08d8490264e411ea5e60490b33d09c5) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:36.761947 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39aa6f4-c325-4829-9240-21eea4fda5fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:36.763407 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39aa6f4-c325-4829-9240-21eea4fda5fb None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:36.769172 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 552/1130] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:08:36 2022] GET /identity/v3/role_assignments?user.id=f08d8490264e411ea5e60490b33d09c5&scope.project.id=2a47193e470c44c9988a1094aa4dddc3&role.id=199fd3093a824ba585b1135c8bf363be => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:38.790787 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0f45e-771e-461b-8aad-25f97c582e33 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:38.791323 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0f45e-771e-461b-8aad-25f97c582e33 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:38.791323 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0f45e-771e-461b-8aad-25f97c582e33 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:38.792625 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 579/1131] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:08:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:08:38.797823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760ec0c-f531-47b4-a481-606df0dfc854 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:38.798219 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760ec0c-f531-47b4-a481-606df0dfc854 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:38.798363 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760ec0c-f531-47b4-a481-606df0dfc854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:38.816264 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-4760ec0c-f531-47b4-a481-606df0dfc854 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:08:38.817467 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-4760ec0c-f531-47b4-a481-606df0dfc854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:08:38.880183 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 553/1132] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:08:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:08:38.903274 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf0f45e-771e-461b-8aad-25f97c582e33 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:38.946265 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:39.001052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf0f45e-771e-461b-8aad-25f97c582e33 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:39.033418 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5761d2-44a1-4dbc-bfed-54d2337c4dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:39.034036 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5761d2-44a1-4dbc-bfed-54d2337c4dcb None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:39.034312 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5761d2-44a1-4dbc-bfed-54d2337c4dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:39.034583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5761d2-44a1-4dbc-bfed-54d2337c4dcb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:39.035718 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5761d2-44a1-4dbc-bfed-54d2337c4dcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:39.037158 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5761d2-44a1-4dbc-bfed-54d2337c4dcb None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:39.038713 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 580/1133] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:08:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:39.044524 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760ec0c-f531-47b4-a481-606df0dfc854 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:39.071665 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:39.140207 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760ec0c-f531-47b4-a481-606df0dfc854 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:39.173533 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a8cef4-16c4-46a0-8c10-30492d4a499f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:39.174144 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8cef4-16c4-46a0-8c10-30492d4a499f None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:39.174422 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8cef4-16c4-46a0-8c10-30492d4a499f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:39.174694 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8cef4-16c4-46a0-8c10-30492d4a499f None admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:39.175366 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a8cef4-16c4-46a0-8c10-30492d4a499f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:39.176805 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a8cef4-16c4-46a0-8c10-30492d4a499f None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:39.282185 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 554/1134] 10.222.0.26 () {60 vars in 1141 bytes} [Fri Aug 5 14:08:39 2022] POST /identity/v3/projects => generated 311 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:08:41.332740 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaef202-8852-4618-bb65-fbb42dd0521f None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:41.333275 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaef202-8852-4618-bb65-fbb42dd0521f None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:41.333275 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaef202-8852-4618-bb65-fbb42dd0521f None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:41.334611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 581/1135] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:08:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:08:41.339815 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b7a63-b21b-44d6-a2d1-9e20accdf164 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:41.340245 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b7a63-b21b-44d6-a2d1-9e20accdf164 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:41.340338 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b7a63-b21b-44d6-a2d1-9e20accdf164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:41.358353 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-cd9b7a63-b21b-44d6-a2d1-9e20accdf164 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:08:41.360559 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-cd9b7a63-b21b-44d6-a2d1-9e20accdf164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:08:41.425885 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 555/1136] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:08:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:08:41.450329 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaef202-8852-4618-bb65-fbb42dd0521f None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:41.549275 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaef202-8852-4618-bb65-fbb42dd0521f None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:41.560038 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:41.582522 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b7bf82-6642-4389-bd5a-6fa4b8d8e764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:41.583197 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7bf82-6642-4389-bd5a-6fa4b8d8e764 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:41.583420 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7bf82-6642-4389-bd5a-6fa4b8d8e764 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:41.583684 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7bf82-6642-4389-bd5a-6fa4b8d8e764 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:41.584824 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b7bf82-6642-4389-bd5a-6fa4b8d8e764 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:41.586220 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b7bf82-6642-4389-bd5a-6fa4b8d8e764 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:41.587770 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 582/1137] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:08:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:41.593544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b7a63-b21b-44d6-a2d1-9e20accdf164 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:41.605358 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:41.695608 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b7a63-b21b-44d6-a2d1-9e20accdf164 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:41.728456 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b3dd1e-bb1f-4c64-b02d-6201b7cbaf4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:41.729060 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3dd1e-bb1f-4c64-b02d-6201b7cbaf4b None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:41.729333 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3dd1e-bb1f-4c64-b02d-6201b7cbaf4b None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:41.729597 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3dd1e-bb1f-4c64-b02d-6201b7cbaf4b None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:41.730271 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b3dd1e-bb1f-4c64-b02d-6201b7cbaf4b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:41.731666 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b3dd1e-bb1f-4c64-b02d-6201b7cbaf4b None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:41.843362 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 556/1138] 10.222.0.26 () {60 vars in 1132 bytes} [Fri Aug 5 14:08:41 2022] POST /identity/v3/users => generated 286 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:08:43.883461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579863bb-3e44-474b-8cec-407d0b9c87bf None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:43.883991 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579863bb-3e44-474b-8cec-407d0b9c87bf None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:43.883991 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579863bb-3e44-474b-8cec-407d0b9c87bf None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:43.885360 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 583/1139] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:08:43.891000 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d25e6c0-b43a-4bb7-bd8e-ccbd86599705 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:43.891419 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d25e6c0-b43a-4bb7-bd8e-ccbd86599705 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:43.891569 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d25e6c0-b43a-4bb7-bd8e-ccbd86599705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:43.909041 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-9d25e6c0-b43a-4bb7-bd8e-ccbd86599705 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:08:43.910223 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-9d25e6c0-b43a-4bb7-bd8e-ccbd86599705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:08:43.967399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:43.975045 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 557/1140] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:08:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 14:08:43.998153 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579863bb-3e44-474b-8cec-407d0b9c87bf None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:44.096189 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579863bb-3e44-474b-8cec-407d0b9c87bf None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:44.128615 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae760b40-bb96-43c9-a812-27971ceeb409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:44.129232 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae760b40-bb96-43c9-a812-27971ceeb409 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:44.129505 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae760b40-bb96-43c9-a812-27971ceeb409 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:44.129769 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae760b40-bb96-43c9-a812-27971ceeb409 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:44.139438 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae760b40-bb96-43c9-a812-27971ceeb409 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:08:44.144394 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-ae760b40-bb96-43c9-a812-27971ceeb409 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:08:44.144394 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:44.144394 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:44.144394 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:44.144394 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:44.144394 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:44.144394 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:44.144394 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:44.144394 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:44.144394 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:44.144394 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:44.144394 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:44.144394 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:08:44.144394 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:08:44.144394 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:08:44.144394 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:08:44.145611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:08:44.145611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:08:44.145611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:08:44.145611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:08:44.145611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:44.145611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:44.145611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:44.145611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:44.145611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:44.145611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:44.145611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:44.145611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:44.145611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:44.145611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:44.145611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:44.145611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:44.146725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:44.146725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:44.146725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:44.146725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:44.146725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:08:44.146725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:08:44.146725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:08:44.146725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:08:44.146725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:08:44.146725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:08:44.146725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:08:44.146725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:44.146725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 584/1141] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Aug 5 14:08:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:08:44.151415 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d25e6c0-b43a-4bb7-bd8e-ccbd86599705 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:44.245589 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d25e6c0-b43a-4bb7-bd8e-ccbd86599705 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:44.277633 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c7f82-695d-404c-9204-d87bfb3eb2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:44.278231 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c7f82-695d-404c-9204-d87bfb3eb2d8 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:44.278501 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c7f82-695d-404c-9204-d87bfb3eb2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:44.278765 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c7f82-695d-404c-9204-d87bfb3eb2d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:44.279464 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03c7f82-695d-404c-9204-d87bfb3eb2d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:44.279763 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03c7f82-695d-404c-9204-d87bfb3eb2d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:44.281709 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03c7f82-695d-404c-9204-d87bfb3eb2d8 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:44.289312 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 558/1142] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:08:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:44.294831 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae760b40-bb96-43c9-a812-27971ceeb409 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:44.333161 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:44.390085 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae760b40-bb96-43c9-a812-27971ceeb409 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:44.422689 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0138db-60fd-4ae5-af49-bb0482de9813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:44.423300 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0138db-60fd-4ae5-af49-bb0482de9813 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:44.423571 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0138db-60fd-4ae5-af49-bb0482de9813 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:44.423836 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0138db-60fd-4ae5-af49-bb0482de9813 None admin] PATH_INFO: `/v3/users/b1846249c2d94448bdbe23d729cca53c` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:44.436190 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0138db-60fd-4ae5-af49-bb0482de9813 None admin] RBAC: Authorizing `identity:get_user(user_id=b1846249c2d94448bdbe23d729cca53c)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:44.437664 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0138db-60fd-4ae5-af49-bb0482de9813 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:44.439228 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 585/1143] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:08:44 2022] GET /identity/v3/users/b1846249c2d94448bdbe23d729cca53c => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:44.444920 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c7f82-695d-404c-9204-d87bfb3eb2d8 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:44.501831 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:44.542585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c7f82-695d-404c-9204-d87bfb3eb2d8 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:44.574465 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546126d1-60ae-4033-9861-55a7a64e1f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:44.575068 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546126d1-60ae-4033-9861-55a7a64e1f44 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:44.575344 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546126d1-60ae-4033-9861-55a7a64e1f44 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:44.575611 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546126d1-60ae-4033-9861-55a7a64e1f44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:44.641802 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546126d1-60ae-4033-9861-55a7a64e1f44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:44.643174 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546126d1-60ae-4033-9861-55a7a64e1f44 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:44.696963 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:44.737835 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 559/1144] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:08:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 14:08:44.744205 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0138db-60fd-4ae5-af49-bb0482de9813 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:44.835795 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:44.841388 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0138db-60fd-4ae5-af49-bb0482de9813 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:44.873547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0256357-8087-41b2-a3e4-70accdc77936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:44.874172 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0256357-8087-41b2-a3e4-70accdc77936 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:44.874447 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0256357-8087-41b2-a3e4-70accdc77936 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:44.875221 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0256357-8087-41b2-a3e4-70accdc77936 None admin] PATH_INFO: `/v3/projects/a5d5660a46624f02a7ef69abb0eb6403` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:44.876449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0256357-8087-41b2-a3e4-70accdc77936 None admin] RBAC: Authorizing `identity:get_project(project_id=a5d5660a46624f02a7ef69abb0eb6403)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:44.877864 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0256357-8087-41b2-a3e4-70accdc77936 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:44.879549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 586/1145] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:08:44 2022] GET /identity/v3/projects/a5d5660a46624f02a7ef69abb0eb6403 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:44.885710 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546126d1-60ae-4033-9861-55a7a64e1f44 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:44.982192 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546126d1-60ae-4033-9861-55a7a64e1f44 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:45.016574 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05129a7d-4618-473d-bfa0-0559a42cb630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:45.017173 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05129a7d-4618-473d-bfa0-0559a42cb630 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:45.017449 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05129a7d-4618-473d-bfa0-0559a42cb630 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:45.017721 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05129a7d-4618-473d-bfa0-0559a42cb630 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:45.018492 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05129a7d-4618-473d-bfa0-0559a42cb630 None admin] RBAC: Adding query filter params (role.id=299c794c7acb4a5ca36288c3191aad10, scope.project.id=a5d5660a46624f02a7ef69abb0eb6403, user.id=b1846249c2d94448bdbe23d729cca53c) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:45.018795 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05129a7d-4618-473d-bfa0-0559a42cb630 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:45.020193 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05129a7d-4618-473d-bfa0-0559a42cb630 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:45.025406 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 560/1146] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:08:44 2022] GET /identity/v3/role_assignments?user.id=b1846249c2d94448bdbe23d729cca53c&scope.project.id=a5d5660a46624f02a7ef69abb0eb6403&role.id=299c794c7acb4a5ca36288c3191aad10 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:47.008379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84686ea0-09ef-4214-8d35-e3759ddb76f0 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:47.008959 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84686ea0-09ef-4214-8d35-e3759ddb76f0 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:47.009070 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84686ea0-09ef-4214-8d35-e3759ddb76f0 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:47.010351 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 587/1147] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:08:47.015524 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377c1a6-0134-4287-86f5-454e1de862f5 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:47.015901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377c1a6-0134-4287-86f5-454e1de862f5 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:47.016079 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377c1a6-0134-4287-86f5-454e1de862f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:47.034188 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-f377c1a6-0134-4287-86f5-454e1de862f5 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:08:47.035375 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f377c1a6-0134-4287-86f5-454e1de862f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:08:47.051774 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:47.100540 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 561/1148] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:08:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:08:47.123207 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84686ea0-09ef-4214-8d35-e3759ddb76f0 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:47.225581 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84686ea0-09ef-4214-8d35-e3759ddb76f0 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:47.258225 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ea83f-2c30-41ab-8e30-eff76bd4bd07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:47.258899 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ea83f-2c30-41ab-8e30-eff76bd4bd07 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:47.259122 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ea83f-2c30-41ab-8e30-eff76bd4bd07 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:47.259388 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ea83f-2c30-41ab-8e30-eff76bd4bd07 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:47.269553 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8ea83f-2c30-41ab-8e30-eff76bd4bd07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:08:47.274407 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-5c8ea83f-2c30-41ab-8e30-eff76bd4bd07 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:08:47.274407 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:47.274407 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:47.274407 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:47.274407 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:47.274407 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:47.274407 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:47.274407 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:47.274407 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:47.274407 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:47.274407 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:47.274407 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:47.274407 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:08:47.274407 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:08:47.274407 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:08:47.274407 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:08:47.275498 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:08:47.275498 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:08:47.275498 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:08:47.275498 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:08:47.275498 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:47.275498 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:47.275498 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:47.275498 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:47.275498 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:47.275498 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:47.275498 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:47.275498 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:47.275498 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:47.275498 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:47.275498 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:47.275498 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:47.276801 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:47.276801 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:47.276801 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:47.276801 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:47.276801 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:08:47.276801 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:08:47.276801 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:08:47.276801 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:08:47.276801 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:08:47.276801 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:08:47.276801 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:08:47.276801 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:47.276801 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 588/1149] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Aug 5 14:08:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:08:47.282330 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f377c1a6-0134-4287-86f5-454e1de862f5 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:47.376244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f377c1a6-0134-4287-86f5-454e1de862f5 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:47.409515 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a1cc5-c0b1-4aa7-95c9-3ed8d1bc89ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:47.410118 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a1cc5-c0b1-4aa7-95c9-3ed8d1bc89ad None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:47.410395 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a1cc5-c0b1-4aa7-95c9-3ed8d1bc89ad None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:47.410664 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a1cc5-c0b1-4aa7-95c9-3ed8d1bc89ad None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:47.411365 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1a1cc5-c0b1-4aa7-95c9-3ed8d1bc89ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:47.411664 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1a1cc5-c0b1-4aa7-95c9-3ed8d1bc89ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:47.413036 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1a1cc5-c0b1-4aa7-95c9-3ed8d1bc89ad None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:47.420775 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 562/1150] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:08:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:47.426636 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ea83f-2c30-41ab-8e30-eff76bd4bd07 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:47.451172 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:47.524906 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ea83f-2c30-41ab-8e30-eff76bd4bd07 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:47.558403 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f485d0-32ce-406f-ae8c-5ec6801ea1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:47.559024 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f485d0-32ce-406f-ae8c-5ec6801ea1c6 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:47.559296 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f485d0-32ce-406f-ae8c-5ec6801ea1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:47.559561 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f485d0-32ce-406f-ae8c-5ec6801ea1c6 None admin] PATH_INFO: `/v3/users/b1846249c2d94448bdbe23d729cca53c` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:47.560717 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f485d0-32ce-406f-ae8c-5ec6801ea1c6 None admin] RBAC: Authorizing `identity:get_user(user_id=b1846249c2d94448bdbe23d729cca53c)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:47.562162 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f485d0-32ce-406f-ae8c-5ec6801ea1c6 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:47.563997 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 589/1151] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:08:47 2022] GET /identity/v3/users/b1846249c2d94448bdbe23d729cca53c => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:47.569724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a1cc5-c0b1-4aa7-95c9-3ed8d1bc89ad None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:47.585961 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:47.668242 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a1cc5-c0b1-4aa7-95c9-3ed8d1bc89ad None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:47.700795 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b36889-775e-43cd-a26d-d0c44788354f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:47.701421 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b36889-775e-43cd-a26d-d0c44788354f None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:47.701698 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b36889-775e-43cd-a26d-d0c44788354f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:47.701970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b36889-775e-43cd-a26d-d0c44788354f None admin] PATH_INFO: `/v3/projects/a5d5660a46624f02a7ef69abb0eb6403` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:47.703263 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b36889-775e-43cd-a26d-d0c44788354f None admin] RBAC: Authorizing `identity:get_project(project_id=a5d5660a46624f02a7ef69abb0eb6403)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:47.704714 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b36889-775e-43cd-a26d-d0c44788354f None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:47.706354 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 563/1152] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:08:47 2022] GET /identity/v3/projects/a5d5660a46624f02a7ef69abb0eb6403 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:47.712280 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f485d0-32ce-406f-ae8c-5ec6801ea1c6 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:47.800525 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:47.808940 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f485d0-32ce-406f-ae8c-5ec6801ea1c6 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:47.842284 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ab786-dee7-4739-b582-a3869ecd91e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:47.842937 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ab786-dee7-4739-b582-a3869ecd91e0 None admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:47.843216 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ab786-dee7-4739-b582-a3869ecd91e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:47.843617 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ab786-dee7-4739-b582-a3869ecd91e0 None admin] PATH_INFO: `/v3/projects/a5d5660a46624f02a7ef69abb0eb6403/users/b1846249c2d94448bdbe23d729cca53c/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:47.845449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ab786-dee7-4739-b582-a3869ecd91e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=a5d5660a46624f02a7ef69abb0eb6403, user_id=b1846249c2d94448bdbe23d729cca53c, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:47.846864 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ab786-dee7-4739-b582-a3869ecd91e0 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:47.911322 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 590/1153] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Aug 5 14:08:47 2022] PUT /identity/v3/projects/a5d5660a46624f02a7ef69abb0eb6403/users/b1846249c2d94448bdbe23d729cca53c/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:08:49.877013 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4702bd-97be-4d45-b23a-36b813f7cc6f None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:49.878785 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4702bd-97be-4d45-b23a-36b813f7cc6f None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:49.878785 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4702bd-97be-4d45-b23a-36b813f7cc6f None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:49.879472 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 564/1154] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:08:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:08:49.884724 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab3935-9e43-4318-98ea-55885642c541 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:49.885160 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab3935-9e43-4318-98ea-55885642c541 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:49.885279 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab3935-9e43-4318-98ea-55885642c541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:49.903557 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-b3ab3935-9e43-4318-98ea-55885642c541 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:08:49.904782 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-b3ab3935-9e43-4318-98ea-55885642c541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:08:49.968871 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 591/1155] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:08:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:08:49.991744 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4702bd-97be-4d45-b23a-36b813f7cc6f None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:50.070636 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:50.088929 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4702bd-97be-4d45-b23a-36b813f7cc6f None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:50.120580 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eed042-61d7-4eb7-b27f-f015d205773f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:50.121200 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eed042-61d7-4eb7-b27f-f015d205773f None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:50.121474 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eed042-61d7-4eb7-b27f-f015d205773f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:50.121742 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eed042-61d7-4eb7-b27f-f015d205773f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:50.132324 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eed042-61d7-4eb7-b27f-f015d205773f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:08:50.137552 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-91eed042-61d7-4eb7-b27f-f015d205773f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:08:50.137552 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:50.137552 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:50.137552 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:50.137552 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:50.137552 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:50.137552 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:50.137552 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:50.137552 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:50.137552 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:50.137552 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:50.137552 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:50.137552 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:08:50.137552 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:08:50.137552 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:08:50.137552 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:08:50.139291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:08:50.139291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:08:50.139291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:08:50.139291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:08:50.139291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:50.139291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:50.139291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:50.139291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:50.139291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:50.139291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:50.139291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:50.139291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:50.139291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:50.139291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:50.139291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:50.139291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:50.141631 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:50.141631 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:50.141631 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:50.141631 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:50.141631 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:08:50.141631 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:08:50.141631 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:08:50.141631 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:08:50.141631 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:08:50.141631 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:08:50.141631 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:08:50.141631 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:08:50.143167 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 565/1156] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Aug 5 14:08:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:08:50.145165 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab3935-9e43-4318-98ea-55885642c541 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:50.224715 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:50.242900 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab3935-9e43-4318-98ea-55885642c541 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:50.276337 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330306ea-8dd6-4f9c-a194-d146dad3120f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:50.276963 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330306ea-8dd6-4f9c-a194-d146dad3120f None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:50.277235 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330306ea-8dd6-4f9c-a194-d146dad3120f None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:50.277503 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330306ea-8dd6-4f9c-a194-d146dad3120f None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:50.278199 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330306ea-8dd6-4f9c-a194-d146dad3120f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:50.278495 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330306ea-8dd6-4f9c-a194-d146dad3120f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:50.279834 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330306ea-8dd6-4f9c-a194-d146dad3120f None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:50.287973 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 592/1157] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:08:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:50.293524 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eed042-61d7-4eb7-b27f-f015d205773f None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:50.390764 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eed042-61d7-4eb7-b27f-f015d205773f None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:50.423776 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5401e579-d850-4937-b671-dffee282c510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:50.424386 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401e579-d850-4937-b671-dffee282c510 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:50.424691 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401e579-d850-4937-b671-dffee282c510 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:50.424959 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401e579-d850-4937-b671-dffee282c510 None admin] PATH_INFO: `/v3/users/b1846249c2d94448bdbe23d729cca53c` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:50.426089 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5401e579-d850-4937-b671-dffee282c510 None admin] RBAC: Authorizing `identity:get_user(user_id=b1846249c2d94448bdbe23d729cca53c)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:50.427507 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5401e579-d850-4937-b671-dffee282c510 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:50.429084 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 566/1158] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:08:50 2022] GET /identity/v3/users/b1846249c2d94448bdbe23d729cca53c => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:50.434701 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330306ea-8dd6-4f9c-a194-d146dad3120f None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:50.533987 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330306ea-8dd6-4f9c-a194-d146dad3120f None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:50.566542 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de02372-b63f-4763-b111-34bfa16eb410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:50.567140 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de02372-b63f-4763-b111-34bfa16eb410 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:50.567413 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de02372-b63f-4763-b111-34bfa16eb410 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:50.567679 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de02372-b63f-4763-b111-34bfa16eb410 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:50.581161 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:50.631988 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de02372-b63f-4763-b111-34bfa16eb410 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:50.633456 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de02372-b63f-4763-b111-34bfa16eb410 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:50.726397 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 593/1159] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:08:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 14:08:50.732311 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5401e579-d850-4937-b671-dffee282c510 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:50.745120 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:50.828161 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5401e579-d850-4937-b671-dffee282c510 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:50.861218 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d159bb2-ebd3-4564-b611-99e6a5f1e408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:50.861842 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d159bb2-ebd3-4564-b611-99e6a5f1e408 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:50.862116 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d159bb2-ebd3-4564-b611-99e6a5f1e408 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:50.862383 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d159bb2-ebd3-4564-b611-99e6a5f1e408 None admin] PATH_INFO: `/v3/projects/a5d5660a46624f02a7ef69abb0eb6403` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:50.863511 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d159bb2-ebd3-4564-b611-99e6a5f1e408 None admin] RBAC: Authorizing `identity:get_project(project_id=a5d5660a46624f02a7ef69abb0eb6403)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:50.864954 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d159bb2-ebd3-4564-b611-99e6a5f1e408 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:50.866579 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 567/1160] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:08:50 2022] GET /identity/v3/projects/a5d5660a46624f02a7ef69abb0eb6403 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:50.872850 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de02372-b63f-4763-b111-34bfa16eb410 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:50.915831 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:50.966720 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de02372-b63f-4763-b111-34bfa16eb410 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:51.000221 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8511cc8e-6731-42ad-ad4f-411b45aae3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:51.000800 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511cc8e-6731-42ad-ad4f-411b45aae3cb None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:51.001076 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511cc8e-6731-42ad-ad4f-411b45aae3cb None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:51.001344 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511cc8e-6731-42ad-ad4f-411b45aae3cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:51.002124 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8511cc8e-6731-42ad-ad4f-411b45aae3cb None admin] RBAC: Adding query filter params (role.id=299c794c7acb4a5ca36288c3191aad10, scope.project.id=a5d5660a46624f02a7ef69abb0eb6403, user.id=b1846249c2d94448bdbe23d729cca53c) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:51.002502 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8511cc8e-6731-42ad-ad4f-411b45aae3cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:51.003912 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8511cc8e-6731-42ad-ad4f-411b45aae3cb None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:51.009569 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 594/1161] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:08:50 2022] GET /identity/v3/role_assignments?user.id=b1846249c2d94448bdbe23d729cca53c&scope.project.id=a5d5660a46624f02a7ef69abb0eb6403&role.id=299c794c7acb4a5ca36288c3191aad10 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:53.040730 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff9adc-759c-4143-8890-56349069183d None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:53.041263 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff9adc-759c-4143-8890-56349069183d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:53.041263 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff9adc-759c-4143-8890-56349069183d None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:53.042603 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 568/1162] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:08:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:08:53.048112 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d68d8-c69d-44c5-955a-779d8a7ab179 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:53.048565 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d68d8-c69d-44c5-955a-779d8a7ab179 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:53.048694 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d68d8-c69d-44c5-955a-779d8a7ab179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:53.067023 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-787d68d8-c69d-44c5-955a-779d8a7ab179 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:08:53.068207 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-787d68d8-c69d-44c5-955a-779d8a7ab179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:08:53.132233 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 595/1163] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:08:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:08:53.154875 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ff9adc-759c-4143-8890-56349069183d None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:53.236968 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:53.259179 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ff9adc-759c-4143-8890-56349069183d None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:53.291663 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ff481-f150-4400-9527-a4d4426a9656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:53.292305 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ff481-f150-4400-9527-a4d4426a9656 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:53.292614 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ff481-f150-4400-9527-a4d4426a9656 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:53.292887 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ff481-f150-4400-9527-a4d4426a9656 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:53.357287 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ff481-f150-4400-9527-a4d4426a9656 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:53.359245 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ff481-f150-4400-9527-a4d4426a9656 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:53.432480 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:53.453891 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 569/1164] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:08:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 14:08:53.459912 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d68d8-c69d-44c5-955a-779d8a7ab179 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:53.498907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:53.555942 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d68d8-c69d-44c5-955a-779d8a7ab179 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:53.588099 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c1bde-fe76-4ad6-b7b3-81dc4791f5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:53.588750 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c1bde-fe76-4ad6-b7b3-81dc4791f5d7 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:53.589029 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c1bde-fe76-4ad6-b7b3-81dc4791f5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:53.589482 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c1bde-fe76-4ad6-b7b3-81dc4791f5d7 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:53.596676 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c1bde-fe76-4ad6-b7b3-81dc4791f5d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:53.598087 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c1bde-fe76-4ad6-b7b3-81dc4791f5d7 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:53.610141 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-533c1bde-fe76-4ad6-b7b3-81dc4791f5d7 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 05 14:08:53.610141 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:53.610141 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 05 14:08:53.610141 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 05 14:08:53.610141 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:08:53.610141 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:08:53.610141 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:08:53.610141 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:08:53.610141 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 05 14:08:53.610141 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:53.610141 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 14:08:53.610141 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:53.610141 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:08:53.610141 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:08:53.610141 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:08:53.610141 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:08:53.610141 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:08:53.611359 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:08:53.611359 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:08:53.611359 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:08:53.611359 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:08:53.611359 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:08:53.611359 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:08:53.611359 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:08:53.611359 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:08:53.611359 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:08:53.611359 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:08:53.611359 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:08:53.611359 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:08:53.611359 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:08:53.611359 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:08:53.611359 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:08:53.611359 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:08:53.612229 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:08:53.612229 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:08:53.612229 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:08:53.612229 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:08:53.612229 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:08:53.612229 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:08:53.612229 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:08:53.612229 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:08:53.612229 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:08:53.612229 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:08:53.612229 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 05 14:08:53.612229 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:08:53.612229 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 05 14:08:53.612229 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:08:53.612229 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 596/1165] 10.222.0.26 () {56 vars in 1117 bytes} [Fri Aug 5 14:08:53 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:08:53.617602 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ff481-f150-4400-9527-a4d4426a9656 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:53.712244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ff481-f150-4400-9527-a4d4426a9656 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:53.745628 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22afb12f-a2c0-4116-95c5-576fd6b35354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:53.746223 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22afb12f-a2c0-4116-95c5-576fd6b35354 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:53.746496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22afb12f-a2c0-4116-95c5-576fd6b35354 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:53.746764 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22afb12f-a2c0-4116-95c5-576fd6b35354 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:53.747437 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22afb12f-a2c0-4116-95c5-576fd6b35354 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:08:53.747735 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22afb12f-a2c0-4116-95c5-576fd6b35354 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:53.749102 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22afb12f-a2c0-4116-95c5-576fd6b35354 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:53.756047 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 570/1166] 10.222.0.26 () {56 vars in 1115 bytes} [Fri Aug 5 14:08:53 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:55.743110 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1a8b9-7c4d-4be0-b39e-55d91091b012 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:55.743671 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1a8b9-7c4d-4be0-b39e-55d91091b012 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:55.743671 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1a8b9-7c4d-4be0-b39e-55d91091b012 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:55.745029 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 597/1167] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:08:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:08:55.750342 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19034d-f00d-4fe4-ae3c-8d53d5771505 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:55.750736 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19034d-f00d-4fe4-ae3c-8d53d5771505 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:55.750890 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19034d-f00d-4fe4-ae3c-8d53d5771505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:55.769181 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-7d19034d-f00d-4fe4-ae3c-8d53d5771505 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:08:55.770367 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-7d19034d-f00d-4fe4-ae3c-8d53d5771505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:08:55.804027 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:55.834532 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 571/1168] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:08:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:08:55.857586 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee1a8b9-7c4d-4be0-b39e-55d91091b012 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:55.953027 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:55.957844 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee1a8b9-7c4d-4be0-b39e-55d91091b012 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:55.990203 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e50f3-37a7-46a7-954e-8825bda1f41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:55.990808 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e50f3-37a7-46a7-954e-8825bda1f41d None admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:55.991081 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e50f3-37a7-46a7-954e-8825bda1f41d None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:55.991351 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e50f3-37a7-46a7-954e-8825bda1f41d None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:55.992061 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45e50f3-37a7-46a7-954e-8825bda1f41d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:55.993682 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45e50f3-37a7-46a7-954e-8825bda1f41d None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:56.060628 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 598/1169] 10.222.0.26 () {60 vars in 1138 bytes} [Fri Aug 5 14:08:55 2022] POST /identity/v3/domains => generated 267 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:08:58.067727 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25926bab-7235-4db5-8b5e-7269e1e97744 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:58.068256 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25926bab-7235-4db5-8b5e-7269e1e97744 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:58.068256 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25926bab-7235-4db5-8b5e-7269e1e97744 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:58.069642 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 572/1170] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:08:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:08:58.074918 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e39e9-becf-4620-9ea1-ca1ebd949505 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:58.075319 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e39e9-becf-4620-9ea1-ca1ebd949505 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:58.075460 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e39e9-becf-4620-9ea1-ca1ebd949505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:58.093509 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-9b5e39e9-becf-4620-9ea1-ca1ebd949505 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:08:58.094681 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-9b5e39e9-becf-4620-9ea1-ca1ebd949505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:08:58.158873 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 599/1171] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:08:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 14:08:58.182445 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25926bab-7235-4db5-8b5e-7269e1e97744 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:58.279822 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25926bab-7235-4db5-8b5e-7269e1e97744 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:58.312670 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfcfb48-9ed6-4090-8921-4e097cb70678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:58.313284 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfcfb48-9ed6-4090-8921-4e097cb70678 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:58.313562 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfcfb48-9ed6-4090-8921-4e097cb70678 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:58.313830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfcfb48-9ed6-4090-8921-4e097cb70678 None admin] PATH_INFO: `/v3/domains/01412201b80246c0988259c70de09218` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:58.323758 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfcfb48-9ed6-4090-8921-4e097cb70678 None admin] RBAC: Authorizing `identity:get_domain(domain_id=01412201b80246c0988259c70de09218)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:58.325183 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfcfb48-9ed6-4090-8921-4e097cb70678 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:58.326746 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 573/1172] 10.222.0.26 () {56 vars in 1183 bytes} [Fri Aug 5 14:08:58 2022] GET /identity/v3/domains/01412201b80246c0988259c70de09218 => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:08:58.332593 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e39e9-becf-4620-9ea1-ca1ebd949505 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:08:58.415564 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:08:58.426865 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e39e9-becf-4620-9ea1-ca1ebd949505 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:08:58.459968 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525d59d-98be-4813-96a6-a83ec9a47f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:08:58.460602 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525d59d-98be-4813-96a6-a83ec9a47f00 None admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:08:58.460882 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525d59d-98be-4813-96a6-a83ec9a47f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:08:58.461153 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525d59d-98be-4813-96a6-a83ec9a47f00 None admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:08:58.461822 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a525d59d-98be-4813-96a6-a83ec9a47f00 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:08:58.463262 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a525d59d-98be-4813-96a6-a83ec9a47f00 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:08:58.550084 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 600/1173] 10.222.0.26 () {60 vars in 1141 bytes} [Fri Aug 5 14:08:58 2022] POST /identity/v3/projects => generated 361 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:09:00.573354 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86ffd7-2c5a-4b97-945b-414074a05dd7 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:09:00.573804 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86ffd7-2c5a-4b97-945b-414074a05dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:09:00.573896 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86ffd7-2c5a-4b97-945b-414074a05dd7 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:09:00.575234 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 574/1174] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:09:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:09:00.580931 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653ced8-a4f0-439c-b281-ea82681f9076 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:09:00.581360 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653ced8-a4f0-439c-b281-ea82681f9076 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:09:00.581495 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653ced8-a4f0-439c-b281-ea82681f9076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:09:00.599318 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-b653ced8-a4f0-439c-b281-ea82681f9076 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:09:00.600536 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-b653ced8-a4f0-439c-b281-ea82681f9076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:09:00.665592 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 601/1175] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:09:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:09:00.690698 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f86ffd7-2c5a-4b97-945b-414074a05dd7 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:09:00.722106 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:09:00.790366 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f86ffd7-2c5a-4b97-945b-414074a05dd7 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:09:00.823783 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ad837-2215-4305-adbe-00494901b504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:09:00.824404 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ad837-2215-4305-adbe-00494901b504 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:09:00.824856 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ad837-2215-4305-adbe-00494901b504 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:09:00.825129 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ad837-2215-4305-adbe-00494901b504 None admin] PATH_INFO: `/v3/domains/01412201b80246c0988259c70de09218` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:09:00.826259 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2ad837-2215-4305-adbe-00494901b504 None admin] RBAC: Authorizing `identity:get_domain(domain_id=01412201b80246c0988259c70de09218)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:09:00.827674 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2ad837-2215-4305-adbe-00494901b504 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:09:00.829249 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 575/1176] 10.222.0.26 () {56 vars in 1183 bytes} [Fri Aug 5 14:09:00 2022] GET /identity/v3/domains/01412201b80246c0988259c70de09218 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:09:00.834904 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b653ced8-a4f0-439c-b281-ea82681f9076 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:09:00.900845 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:09:00.930364 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b653ced8-a4f0-439c-b281-ea82681f9076 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:09:00.963066 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0617d547-f152-493b-8c83-18fb3243e846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:09:00.963667 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617d547-f152-493b-8c83-18fb3243e846 None admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:09:00.963946 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617d547-f152-493b-8c83-18fb3243e846 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:09:00.964218 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617d547-f152-493b-8c83-18fb3243e846 None admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:09:00.964938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0617d547-f152-493b-8c83-18fb3243e846 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:09:00.966354 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0617d547-f152-493b-8c83-18fb3243e846 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:09:01.037489 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 602/1177] 10.222.0.26 () {60 vars in 1132 bytes} [Fri Aug 5 14:09:00 2022] POST /identity/v3/users => generated 311 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:09:03.053480 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7e876-2bb4-42fd-b578-3b42877fca7a None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:09:03.054007 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7e876-2bb4-42fd-b578-3b42877fca7a None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:09:03.054097 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7e876-2bb4-42fd-b578-3b42877fca7a None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:09:03.055366 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 576/1178] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:09:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:09:03.060638 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72912f82-7f72-4561-a6a9-d1bb975c2bef None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:09:03.061008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72912f82-7f72-4561-a6a9-d1bb975c2bef None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:09:03.061204 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72912f82-7f72-4561-a6a9-d1bb975c2bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:09:03.079450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-72912f82-7f72-4561-a6a9-d1bb975c2bef None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:09:03.080669 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-72912f82-7f72-4561-a6a9-d1bb975c2bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:09:03.145601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 603/1179] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:09:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:09:03.168842 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b7e876-2bb4-42fd-b578-3b42877fca7a None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:09:03.262988 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:09:03.267138 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b7e876-2bb4-42fd-b578-3b42877fca7a None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:09:03.304602 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73fede-2ba7-449f-b7e1-fd91122e42bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:09:03.305332 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73fede-2ba7-449f-b7e1-fd91122e42bc None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:09:03.305831 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73fede-2ba7-449f-b7e1-fd91122e42bc None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:09:03.306151 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73fede-2ba7-449f-b7e1-fd91122e42bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:09:03.316712 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f73fede-2ba7-449f-b7e1-fd91122e42bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:09:03.321578 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-1f73fede-2ba7-449f-b7e1-fd91122e42bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:09:03.321578 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:09:03.321578 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:09:03.321578 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:09:03.321578 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:09:03.321578 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:09:03.321578 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:09:03.321578 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:09:03.321578 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:09:03.321578 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:09:03.321578 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:09:03.321578 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:09:03.321578 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:09:03.321578 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:09:03.321578 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:09:03.321578 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:09:03.322706 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:09:03.322706 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:09:03.322706 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:09:03.322706 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:09:03.322706 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:09:03.322706 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:09:03.322706 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:09:03.322706 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:09:03.322706 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:09:03.322706 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:09:03.322706 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:09:03.322706 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:09:03.322706 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:09:03.322706 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:09:03.322706 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:09:03.322706 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:09:03.323613 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:09:03.323613 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:09:03.323613 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:09:03.323613 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:09:03.323613 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:09:03.323613 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:09:03.323613 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:09:03.323613 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:09:03.323613 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:09:03.323613 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:09:03.323613 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:09:03.323613 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:09:03.323613 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 577/1180] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Aug 5 14:09:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:09:03.329284 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72912f82-7f72-4561-a6a9-d1bb975c2bef None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:09:03.382469 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:09:03.423790 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72912f82-7f72-4561-a6a9-d1bb975c2bef None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:09:03.457426 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d656c-69c2-491b-9cfc-07423bfa097e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:09:03.458024 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d656c-69c2-491b-9cfc-07423bfa097e None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:09:03.458295 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d656c-69c2-491b-9cfc-07423bfa097e None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:09:03.458561 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d656c-69c2-491b-9cfc-07423bfa097e None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:09:03.459258 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d656c-69c2-491b-9cfc-07423bfa097e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:09:03.459559 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d656c-69c2-491b-9cfc-07423bfa097e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:09:03.460936 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d656c-69c2-491b-9cfc-07423bfa097e None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:09:03.468705 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 604/1181] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:09:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:09:03.474347 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73fede-2ba7-449f-b7e1-fd91122e42bc None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:09:03.573362 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73fede-2ba7-449f-b7e1-fd91122e42bc None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:09:03.606530 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f14f8eb-fe91-44ef-8d52-b1d6246161f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:09:03.607157 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14f8eb-fe91-44ef-8d52-b1d6246161f4 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:09:03.607437 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14f8eb-fe91-44ef-8d52-b1d6246161f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:09:03.607709 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14f8eb-fe91-44ef-8d52-b1d6246161f4 None admin] PATH_INFO: `/v3/users/aceb6a8403d346d698dbd9b870e2ccec` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:09:03.621418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f14f8eb-fe91-44ef-8d52-b1d6246161f4 None admin] RBAC: Authorizing `identity:get_user(user_id=aceb6a8403d346d698dbd9b870e2ccec)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:09:03.622852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f14f8eb-fe91-44ef-8d52-b1d6246161f4 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:09:03.624819 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 578/1182] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:09:03 2022] GET /identity/v3/users/aceb6a8403d346d698dbd9b870e2ccec => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:09:03.630675 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d656c-69c2-491b-9cfc-07423bfa097e None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:09:03.727648 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d656c-69c2-491b-9cfc-07423bfa097e None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:09:03.744801 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:09:03.760655 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057cf0c-a182-4525-8a01-26f417f0d14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:09:03.761367 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057cf0c-a182-4525-8a01-26f417f0d14f None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:09:03.761640 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057cf0c-a182-4525-8a01-26f417f0d14f None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:09:03.761907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057cf0c-a182-4525-8a01-26f417f0d14f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:09:03.828198 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9057cf0c-a182-4525-8a01-26f417f0d14f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:09:03.829604 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9057cf0c-a182-4525-8a01-26f417f0d14f None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:09:03.924295 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 605/1183] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:09:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 14:09:03.930185 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f14f8eb-fe91-44ef-8d52-b1d6246161f4 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:09:03.948540 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:09:04.026288 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f14f8eb-fe91-44ef-8d52-b1d6246161f4 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:09:04.059874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0396264-d3f5-4c3d-893c-4dc0e8273fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:09:04.060519 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0396264-d3f5-4c3d-893c-4dc0e8273fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:09:04.060794 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0396264-d3f5-4c3d-893c-4dc0e8273fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:09:04.061062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0396264-d3f5-4c3d-893c-4dc0e8273fd7 None admin] PATH_INFO: `/v3/projects/bb859f354ecf42df8232ac8664abb08c` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:09:04.062653 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0396264-d3f5-4c3d-893c-4dc0e8273fd7 None admin] RBAC: Authorizing `identity:get_project(project_id=bb859f354ecf42df8232ac8664abb08c)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:09:04.064061 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0396264-d3f5-4c3d-893c-4dc0e8273fd7 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:09:04.065708 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 579/1184] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:09:03 2022] GET /identity/v3/projects/bb859f354ecf42df8232ac8664abb08c => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:09:04.071660 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057cf0c-a182-4525-8a01-26f417f0d14f None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:09:04.090218 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:09:04.169559 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057cf0c-a182-4525-8a01-26f417f0d14f None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:09:04.203588 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e72f6b3-5ff6-4042-b0fe-1fa76936c1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:09:04.204169 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72f6b3-5ff6-4042-b0fe-1fa76936c1f1 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:09:04.204464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72f6b3-5ff6-4042-b0fe-1fa76936c1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:09:04.204734 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72f6b3-5ff6-4042-b0fe-1fa76936c1f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:09:04.205502 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e72f6b3-5ff6-4042-b0fe-1fa76936c1f1 None admin] RBAC: Adding query filter params (role.id=299c794c7acb4a5ca36288c3191aad10, scope.project.id=bb859f354ecf42df8232ac8664abb08c, user.id=aceb6a8403d346d698dbd9b870e2ccec) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:09:04.205805 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e72f6b3-5ff6-4042-b0fe-1fa76936c1f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:09:04.207196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e72f6b3-5ff6-4042-b0fe-1fa76936c1f1 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:09:04.212312 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 606/1185] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:09:04 2022] GET /identity/v3/role_assignments?user.id=aceb6a8403d346d698dbd9b870e2ccec&scope.project.id=bb859f354ecf42df8232ac8664abb08c&role.id=299c794c7acb4a5ca36288c3191aad10 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:09:06.166263 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e5367-720d-409b-b95c-3d75c011a210 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:09:06.166865 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e5367-720d-409b-b95c-3d75c011a210 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:09:06.166865 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e5367-720d-409b-b95c-3d75c011a210 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:09:06.168140 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 580/1186] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:09:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:09:06.173782 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00102a0-77e8-45c5-b572-9984403a6384 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:09:06.174219 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00102a0-77e8-45c5-b572-9984403a6384 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:09:06.174326 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00102a0-77e8-45c5-b572-9984403a6384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:09:06.192509 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-c00102a0-77e8-45c5-b572-9984403a6384 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:09:06.195765 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-c00102a0-77e8-45c5-b572-9984403a6384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:09:06.257586 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 607/1187] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:09:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:09:06.279527 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e5367-720d-409b-b95c-3d75c011a210 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:09:06.361145 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:09:06.376696 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e5367-720d-409b-b95c-3d75c011a210 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:09:06.409527 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc53fda-0e4e-4462-b1d3-7e10a5a114ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:09:06.410140 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc53fda-0e4e-4462-b1d3-7e10a5a114ce None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:09:06.410412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc53fda-0e4e-4462-b1d3-7e10a5a114ce None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:09:06.410679 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc53fda-0e4e-4462-b1d3-7e10a5a114ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:09:06.420326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc53fda-0e4e-4462-b1d3-7e10a5a114ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:09:06.425521 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-4dc53fda-0e4e-4462-b1d3-7e10a5a114ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:09:06.425521 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:09:06.425521 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:09:06.425521 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:09:06.425521 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:09:06.425521 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:09:06.425521 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:09:06.425521 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:09:06.425521 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:09:06.425521 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:09:06.425521 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:09:06.425521 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:09:06.425521 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:09:06.425521 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:09:06.425521 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:09:06.425521 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:09:06.426707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:09:06.426707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:09:06.426707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:09:06.426707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:09:06.426707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:09:06.426707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:09:06.426707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:09:06.426707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:09:06.426707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:09:06.426707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:09:06.426707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:09:06.426707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:09:06.426707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:09:06.426707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:09:06.426707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:09:06.426707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:09:06.427995 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:09:06.427995 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:09:06.427995 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:09:06.427995 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:09:06.427995 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:09:06.427995 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:09:06.427995 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:09:06.427995 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:09:06.427995 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:09:06.427995 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:09:06.427995 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:09:06.427995 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:09:06.427995 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 581/1188] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Aug 5 14:09:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:09:06.432550 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00102a0-77e8-45c5-b572-9984403a6384 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:09:06.446998 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:09:06.532379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00102a0-77e8-45c5-b572-9984403a6384 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:09:06.565339 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cbdbb9-8fa4-430c-b71b-cfb82ab7e1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:09:06.565946 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbdbb9-8fa4-430c-b71b-cfb82ab7e1d0 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:09:06.566222 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbdbb9-8fa4-430c-b71b-cfb82ab7e1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:09:06.566494 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbdbb9-8fa4-430c-b71b-cfb82ab7e1d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:09:06.567196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cbdbb9-8fa4-430c-b71b-cfb82ab7e1d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:09:06.567495 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cbdbb9-8fa4-430c-b71b-cfb82ab7e1d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:09:06.568884 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cbdbb9-8fa4-430c-b71b-cfb82ab7e1d0 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:09:06.577376 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 608/1189] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:09:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:09:06.583227 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc53fda-0e4e-4462-b1d3-7e10a5a114ce None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:09:06.678734 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc53fda-0e4e-4462-b1d3-7e10a5a114ce None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:09:06.711590 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a32a4-16e5-45a3-b88d-40278009daad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:09:06.712209 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a32a4-16e5-45a3-b88d-40278009daad None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:09:06.712519 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a32a4-16e5-45a3-b88d-40278009daad None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:09:06.712795 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a32a4-16e5-45a3-b88d-40278009daad None admin] PATH_INFO: `/v3/users/aceb6a8403d346d698dbd9b870e2ccec` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:09:06.713936 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14a32a4-16e5-45a3-b88d-40278009daad None admin] RBAC: Authorizing `identity:get_user(user_id=aceb6a8403d346d698dbd9b870e2ccec)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:09:06.715357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14a32a4-16e5-45a3-b88d-40278009daad None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:09:06.717085 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 582/1190] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:09:06 2022] GET /identity/v3/users/aceb6a8403d346d698dbd9b870e2ccec => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:09:06.722892 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cbdbb9-8fa4-430c-b71b-cfb82ab7e1d0 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:09:06.795985 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:09:06.819930 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cbdbb9-8fa4-430c-b71b-cfb82ab7e1d0 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:09:06.852048 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1230644-739b-445a-90bf-a16292e5e43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:09:06.852697 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1230644-739b-445a-90bf-a16292e5e43f None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:09:06.852975 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1230644-739b-445a-90bf-a16292e5e43f None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:09:06.853244 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1230644-739b-445a-90bf-a16292e5e43f None admin] PATH_INFO: `/v3/projects/bb859f354ecf42df8232ac8664abb08c` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:09:06.854380 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1230644-739b-445a-90bf-a16292e5e43f None admin] RBAC: Authorizing `identity:get_project(project_id=bb859f354ecf42df8232ac8664abb08c)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:09:06.855784 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1230644-739b-445a-90bf-a16292e5e43f None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:09:06.857437 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 609/1191] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:09:06 2022] GET /identity/v3/projects/bb859f354ecf42df8232ac8664abb08c => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:09:06.863829 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a32a4-16e5-45a3-b88d-40278009daad None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:09:06.880246 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:09:06.960809 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a32a4-16e5-45a3-b88d-40278009daad None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:09:06.993706 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012663a2-9c5d-4918-977d-2d34a1201d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:09:06.994365 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012663a2-9c5d-4918-977d-2d34a1201d5f None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:09:06.994644 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012663a2-9c5d-4918-977d-2d34a1201d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:09:06.994915 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012663a2-9c5d-4918-977d-2d34a1201d5f None admin] PATH_INFO: `/v3/projects/bb859f354ecf42df8232ac8664abb08c/users/aceb6a8403d346d698dbd9b870e2ccec/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:09:06.996849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012663a2-9c5d-4918-977d-2d34a1201d5f None admin] RBAC: Authorizing `identity:create_grant(project_id=bb859f354ecf42df8232ac8664abb08c, user_id=aceb6a8403d346d698dbd9b870e2ccec, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:09:06.998261 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012663a2-9c5d-4918-977d-2d34a1201d5f None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:09:07.047813 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 583/1192] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Aug 5 14:09:06 2022] PUT /identity/v3/projects/bb859f354ecf42df8232ac8664abb08c/users/aceb6a8403d346d698dbd9b870e2ccec/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:09:09.031298 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e710123-9dee-4282-bd75-eb1d063104bf None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:09:09.031855 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e710123-9dee-4282-bd75-eb1d063104bf None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:09:09.031855 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e710123-9dee-4282-bd75-eb1d063104bf None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:09:09.033208 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 610/1193] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:09:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:09:09.038389 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619b223-c734-4b45-9a55-aa06ef3d4e02 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:09:09.038775 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619b223-c734-4b45-9a55-aa06ef3d4e02 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:09:09.038941 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619b223-c734-4b45-9a55-aa06ef3d4e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:09:09.056566 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-a619b223-c734-4b45-9a55-aa06ef3d4e02 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:09:09.057761 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a619b223-c734-4b45-9a55-aa06ef3d4e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:09:09.122172 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 584/1194] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:09:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:09:09.145947 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e710123-9dee-4282-bd75-eb1d063104bf None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:09:09.244061 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e710123-9dee-4282-bd75-eb1d063104bf None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:09:09.275945 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca0d0d2-9983-4cee-96d5-42bf097c6e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:09:09.276577 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0d0d2-9983-4cee-96d5-42bf097c6e8f None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:09:09.276856 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0d0d2-9983-4cee-96d5-42bf097c6e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:09:09.277126 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0d0d2-9983-4cee-96d5-42bf097c6e8f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:09:09.287924 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca0d0d2-9983-4cee-96d5-42bf097c6e8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:09:09.292784 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-7ca0d0d2-9983-4cee-96d5-42bf097c6e8f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:09:09.292784 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:09:09.292784 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:09:09.292784 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:09:09.292784 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:09:09.292784 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:09:09.292784 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:09:09.292784 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:09:09.292784 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:09:09.292784 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:09:09.292784 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:09:09.292784 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:09:09.292784 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:09:09.292784 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:09:09.292784 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:09:09.292784 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:09:09.293879 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:09:09.293879 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:09:09.293879 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:09:09.293879 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:09:09.293879 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:09:09.293879 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:09:09.293879 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:09:09.293879 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:09:09.293879 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:09:09.293879 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:09:09.293879 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:09:09.293879 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:09:09.293879 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:09:09.293879 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:09:09.293879 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:09:09.293879 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:09:09.294909 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:09:09.294909 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:09:09.294909 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:09:09.294909 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:09:09.294909 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:09:09.294909 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:09:09.294909 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:09:09.294909 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:09:09.294909 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:09:09.294909 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:09:09.294909 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:09:09.294909 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:09:09.294909 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 611/1195] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Aug 5 14:09:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:09:09.299924 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a619b223-c734-4b45-9a55-aa06ef3d4e02 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:09:09.303374 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:09:09.399393 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a619b223-c734-4b45-9a55-aa06ef3d4e02 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:09:09.431318 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf8c9f-4535-4fb4-a7a6-61d78e22a4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:09:09.431925 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf8c9f-4535-4fb4-a7a6-61d78e22a4cf None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:09:09.432204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf8c9f-4535-4fb4-a7a6-61d78e22a4cf None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:09:09.432502 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf8c9f-4535-4fb4-a7a6-61d78e22a4cf None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:09:09.433209 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf8c9f-4535-4fb4-a7a6-61d78e22a4cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:09:09.433510 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf8c9f-4535-4fb4-a7a6-61d78e22a4cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:09:09.434877 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf8c9f-4535-4fb4-a7a6-61d78e22a4cf None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:09:09.443207 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 585/1196] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:09:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:09:09.449024 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca0d0d2-9983-4cee-96d5-42bf097c6e8f None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:09:09.452464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:09:09.546765 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca0d0d2-9983-4cee-96d5-42bf097c6e8f None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:09:09.579648 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf7f12-0088-49fc-bf26-456bdb8f75bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:09:09.580274 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf7f12-0088-49fc-bf26-456bdb8f75bf None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:09:09.580578 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf7f12-0088-49fc-bf26-456bdb8f75bf None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:09:09.580852 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf7f12-0088-49fc-bf26-456bdb8f75bf None admin] PATH_INFO: `/v3/users/aceb6a8403d346d698dbd9b870e2ccec` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:09:09.581990 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cf7f12-0088-49fc-bf26-456bdb8f75bf None admin] RBAC: Authorizing `identity:get_user(user_id=aceb6a8403d346d698dbd9b870e2ccec)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:09:09.583426 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cf7f12-0088-49fc-bf26-456bdb8f75bf None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:09:09.585039 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 612/1197] 10.222.0.26 () {56 vars in 1177 bytes} [Fri Aug 5 14:09:09 2022] GET /identity/v3/users/aceb6a8403d346d698dbd9b870e2ccec => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:09:09.590716 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf8c9f-4535-4fb4-a7a6-61d78e22a4cf None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:09:09.655076 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:09:09.688133 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf8c9f-4535-4fb4-a7a6-61d78e22a4cf None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:09:09.720244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ac62e-c079-4340-9aff-e502bf6a9de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:09:09.720878 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ac62e-c079-4340-9aff-e502bf6a9de8 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:09:09.721152 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ac62e-c079-4340-9aff-e502bf6a9de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:09:09.721417 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ac62e-c079-4340-9aff-e502bf6a9de8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:09:09.785274 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8ac62e-c079-4340-9aff-e502bf6a9de8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:09:09.786694 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8ac62e-c079-4340-9aff-e502bf6a9de8 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:09:09.848143 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:09:09.882508 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 586/1198] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:09:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 14:09:09.888456 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf7f12-0088-49fc-bf26-456bdb8f75bf None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:09:09.955906 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:09:09.985171 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf7f12-0088-49fc-bf26-456bdb8f75bf None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:09:10.018332 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8479c92-28e0-413c-bcdc-b7f2e168f3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:09:10.018951 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8479c92-28e0-413c-bcdc-b7f2e168f3db None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:09:10.019230 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8479c92-28e0-413c-bcdc-b7f2e168f3db None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:09:10.019499 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8479c92-28e0-413c-bcdc-b7f2e168f3db None admin] PATH_INFO: `/v3/projects/bb859f354ecf42df8232ac8664abb08c` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:09:10.020667 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8479c92-28e0-413c-bcdc-b7f2e168f3db None admin] RBAC: Authorizing `identity:get_project(project_id=bb859f354ecf42df8232ac8664abb08c)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:09:10.022071 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8479c92-28e0-413c-bcdc-b7f2e168f3db None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:09:10.023690 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 613/1199] 10.222.0.26 () {56 vars in 1186 bytes} [Fri Aug 5 14:09:09 2022] GET /identity/v3/projects/bb859f354ecf42df8232ac8664abb08c => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:09:10.029887 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ac62e-c079-4340-9aff-e502bf6a9de8 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:09:10.126214 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ac62e-c079-4340-9aff-e502bf6a9de8 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:09:10.158578 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16019c5b-26e7-4cf5-a3f0-b1a402853cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:09:10.159172 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16019c5b-26e7-4cf5-a3f0-b1a402853cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:09:10.159451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16019c5b-26e7-4cf5-a3f0-b1a402853cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:09:10.159727 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16019c5b-26e7-4cf5-a3f0-b1a402853cb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:09:10.160527 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16019c5b-26e7-4cf5-a3f0-b1a402853cb9 None admin] RBAC: Adding query filter params (role.id=299c794c7acb4a5ca36288c3191aad10, scope.project.id=bb859f354ecf42df8232ac8664abb08c, user.id=aceb6a8403d346d698dbd9b870e2ccec) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:09:10.160838 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16019c5b-26e7-4cf5-a3f0-b1a402853cb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:09:10.162256 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16019c5b-26e7-4cf5-a3f0-b1a402853cb9 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:09:10.167834 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 587/1200] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:09:10 2022] GET /identity/v3/role_assignments?user.id=aceb6a8403d346d698dbd9b870e2ccec&scope.project.id=bb859f354ecf42df8232ac8664abb08c&role.id=299c794c7acb4a5ca36288c3191aad10 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:09:20.014295 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=31 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:09:20.165312 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=85 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:21.871446 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29873bb2-c1ff-436c-8f2d-a40b3c0cbb64 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:21.872093 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29873bb2-c1ff-436c-8f2d-a40b3c0cbb64 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:21.872093 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29873bb2-c1ff-436c-8f2d-a40b3c0cbb64 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:21.873394 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 614/1201] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:10:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:10:21.879649 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d88f6d-62a7-414c-ad23-3685ba67a803 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:21.879649 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d88f6d-62a7-414c-ad23-3685ba67a803 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:21.879649 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d88f6d-62a7-414c-ad23-3685ba67a803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:21.896779 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-92d88f6d-62a7-414c-ad23-3685ba67a803 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:10:21.897941 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-92d88f6d-62a7-414c-ad23-3685ba67a803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:10:21.964859 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 588/1202] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:10:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:10:21.989335 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29873bb2-c1ff-436c-8f2d-a40b3c0cbb64 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:22.088490 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29873bb2-c1ff-436c-8f2d-a40b3c0cbb64 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:22.122098 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1b661-9e64-43ee-acaf-512ca4c37a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:22.122652 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1b661-9e64-43ee-acaf-512ca4c37a8d None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:22.122927 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1b661-9e64-43ee-acaf-512ca4c37a8d None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:22.123201 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1b661-9e64-43ee-acaf-512ca4c37a8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:22.133580 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c1b661-9e64-43ee-acaf-512ca4c37a8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:22.135014 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c1b661-9e64-43ee-acaf-512ca4c37a8d None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:22.149462 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-22c1b661-9e64-43ee-acaf-512ca4c37a8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:10:22.149462 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:10:22.149462 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:10:22.149462 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:10:22.149462 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:10:22.149462 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:10:22.149462 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:10:22.149462 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:10:22.149462 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:10:22.149462 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:10:22.149462 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:10:22.149462 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:10:22.149462 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:10:22.149462 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:10:22.149462 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:10:22.150584 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:10:22.150584 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:10:22.150584 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:10:22.150584 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:10:22.150584 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:10:22.150584 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:10:22.150584 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:10:22.150584 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:10:22.150584 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:10:22.150584 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:10:22.150584 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:10:22.150584 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:10:22.150584 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:10:22.150584 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:10:22.150584 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:10:22.150584 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:10:22.150584 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:10:22.152494 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:10:22.152494 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:10:22.152494 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:10:22.152494 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:10:22.152494 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 615/1203] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:10:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:10:22.157380 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d88f6d-62a7-414c-ad23-3685ba67a803 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:22.254464 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d88f6d-62a7-414c-ad23-3685ba67a803 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:22.283140 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:22.288928 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee3fec9-04e8-45cc-bae9-65c01771e4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:22.293454 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3fec9-04e8-45cc-bae9-65c01771e4a4 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:22.293454 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3fec9-04e8-45cc-bae9-65c01771e4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:22.293454 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3fec9-04e8-45cc-bae9-65c01771e4a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:22.293454 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee3fec9-04e8-45cc-bae9-65c01771e4a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:22.293454 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee3fec9-04e8-45cc-bae9-65c01771e4a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:22.293454 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee3fec9-04e8-45cc-bae9-65c01771e4a4 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:22.303883 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 589/1204] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:10:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:22.310196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1b661-9e64-43ee-acaf-512ca4c37a8d None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:22.364818 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:22.414503 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1b661-9e64-43ee-acaf-512ca4c37a8d None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:22.448092 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1a70e5-2fc4-4c0c-81a5-aef324ddc75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:22.449859 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a70e5-2fc4-4c0c-81a5-aef324ddc75e None admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:22.450143 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a70e5-2fc4-4c0c-81a5-aef324ddc75e None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:22.450413 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a70e5-2fc4-4c0c-81a5-aef324ddc75e None admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:22.451078 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1a70e5-2fc4-4c0c-81a5-aef324ddc75e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:22.452614 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1a70e5-2fc4-4c0c-81a5-aef324ddc75e None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:22.545529 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 616/1205] 10.222.0.26 () {60 vars in 1132 bytes} [Fri Aug 5 14:10:22 2022] POST /identity/v3/users => generated 251 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:10:24.682284 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ebb58-e2c3-402a-8549-4c4e30dd62dc None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:24.683312 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ebb58-e2c3-402a-8549-4c4e30dd62dc None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:24.683312 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ebb58-e2c3-402a-8549-4c4e30dd62dc None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:24.684793 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 590/1206] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:10:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:10:24.690028 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e70e0-c6c3-453d-bed7-72006f2307fd None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:24.690437 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e70e0-c6c3-453d-bed7-72006f2307fd None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:24.690619 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e70e0-c6c3-453d-bed7-72006f2307fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:24.711026 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-0d8e70e0-c6c3-453d-bed7-72006f2307fd None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:10:24.712996 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-0d8e70e0-c6c3-453d-bed7-72006f2307fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:10:24.777537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 617/1207] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:10:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 14:10:24.800716 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ebb58-e2c3-402a-8549-4c4e30dd62dc None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:24.899634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ebb58-e2c3-402a-8549-4c4e30dd62dc None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:24.934222 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad7fdc-75cf-4702-aae7-81cbf3e2bb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:24.934853 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad7fdc-75cf-4702-aae7-81cbf3e2bb67 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:24.935130 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad7fdc-75cf-4702-aae7-81cbf3e2bb67 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:24.935401 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad7fdc-75cf-4702-aae7-81cbf3e2bb67 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:24.945112 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ad7fdc-75cf-4702-aae7-81cbf3e2bb67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:10:24.950042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-01ad7fdc-75cf-4702-aae7-81cbf3e2bb67 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:10:24.950042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:10:24.950042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:10:24.950042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:10:24.950042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:10:24.950042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:10:24.950042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:10:24.950042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:10:24.950042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:10:24.950042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:10:24.950042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:10:24.950042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:10:24.950042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:10:24.950042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:10:24.950042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:10:24.950042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:10:24.951014 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:10:24.951014 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:10:24.951014 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:10:24.951014 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:10:24.951014 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:10:24.951014 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:10:24.951014 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:10:24.951014 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:10:24.951014 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:10:24.951014 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:10:24.951014 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:10:24.951014 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:10:24.951014 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:10:24.951014 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:10:24.951014 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:10:24.951014 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:10:24.951912 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:10:24.951912 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:10:24.951912 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:10:24.951912 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:10:24.951912 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:10:24.951912 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:10:24.951912 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:10:24.951912 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:10:24.951912 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:10:24.951912 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:10:24.951912 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:10:24.951912 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:10:24.951912 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 591/1208] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Aug 5 14:10:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:10:24.957329 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e70e0-c6c3-453d-bed7-72006f2307fd None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:24.997252 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:25.055472 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e70e0-c6c3-453d-bed7-72006f2307fd None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:25.087409 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a1352-79f5-49d0-b41d-da7de489b9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:25.088024 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a1352-79f5-49d0-b41d-da7de489b9a3 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:25.088298 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a1352-79f5-49d0-b41d-da7de489b9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:25.088598 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a1352-79f5-49d0-b41d-da7de489b9a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:25.089311 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a1352-79f5-49d0-b41d-da7de489b9a3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:25.089612 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a1352-79f5-49d0-b41d-da7de489b9a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:25.091763 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a1352-79f5-49d0-b41d-da7de489b9a3 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:25.099256 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 618/1209] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Aug 5 14:10:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:25.105157 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad7fdc-75cf-4702-aae7-81cbf3e2bb67 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:25.156256 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:25.204786 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad7fdc-75cf-4702-aae7-81cbf3e2bb67 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:25.238302 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eb6cb7-cf74-4ece-865f-f43cb9d812fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:25.238923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb6cb7-cf74-4ece-865f-f43cb9d812fb None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:25.239195 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb6cb7-cf74-4ece-865f-f43cb9d812fb None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:25.239464 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb6cb7-cf74-4ece-865f-f43cb9d812fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:25.250673 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eb6cb7-cf74-4ece-865f-f43cb9d812fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:25.252581 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eb6cb7-cf74-4ece-865f-f43cb9d812fb None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:25.266377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-24eb6cb7-cf74-4ece-865f-f43cb9d812fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:10:25.266377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:10:25.266377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:10:25.266377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:10:25.266377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:10:25.266377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:10:25.266377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:10:25.266377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:10:25.266377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:10:25.266377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:10:25.266377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:10:25.266377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:10:25.266377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:10:25.266377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:10:25.266377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:10:25.267418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:10:25.267418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:10:25.267418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:10:25.267418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:10:25.267418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:10:25.267418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:10:25.267418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:10:25.267418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:10:25.267418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:10:25.267418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:10:25.267418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:10:25.267418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:10:25.267418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:10:25.267418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:10:25.267418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:10:25.267418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:10:25.267418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:10:25.268567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:10:25.268567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:10:25.268567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:10:25.268567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:10:25.268567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 592/1210] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:10:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:10:25.273379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a1352-79f5-49d0-b41d-da7de489b9a3 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:25.369064 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:25.371937 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a1352-79f5-49d0-b41d-da7de489b9a3 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:25.404580 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f084f-e6ae-46d1-8efe-d840531932f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:25.405192 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f084f-e6ae-46d1-8efe-d840531932f6 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:25.405471 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f084f-e6ae-46d1-8efe-d840531932f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:25.405742 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f084f-e6ae-46d1-8efe-d840531932f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:25.406425 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180f084f-e6ae-46d1-8efe-d840531932f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:25.406727 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180f084f-e6ae-46d1-8efe-d840531932f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:25.408117 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180f084f-e6ae-46d1-8efe-d840531932f6 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:25.419899 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 619/1211] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:10:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:25.425572 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eb6cb7-cf74-4ece-865f-f43cb9d812fb None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:25.524714 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eb6cb7-cf74-4ece-865f-f43cb9d812fb None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:25.525723 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:25.559146 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0fe424-fed5-4a8d-b100-dcbe18b47e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:25.559845 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0fe424-fed5-4a8d-b100-dcbe18b47e2b None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:25.560120 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0fe424-fed5-4a8d-b100-dcbe18b47e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:25.560392 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0fe424-fed5-4a8d-b100-dcbe18b47e2b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:25.567410 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0fe424-fed5-4a8d-b100-dcbe18b47e2b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:25.568880 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0fe424-fed5-4a8d-b100-dcbe18b47e2b None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:25.581682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-3d0fe424-fed5-4a8d-b100-dcbe18b47e2b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 05 14:10:25.581682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:10:25.581682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:10:25.581682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:10:25.581682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:10:25.581682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:10:25.581682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:10:25.581682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:10:25.581682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:10:25.581682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:10:25.581682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:10:25.581682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:10:25.581682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 14:10:25.581682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 14:10:25.581682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 14:10:25.581682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 14:10:25.582901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:10:25.582901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:10:25.582901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:10:25.582901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:10:25.582901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:10:25.582901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:10:25.582901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:10:25.582901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:10:25.582901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:10:25.582901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:10:25.582901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:10:25.582901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:10:25.582901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:10:25.582901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:10:25.582901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:10:25.582901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:10:25.582901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:10:25.582901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:10:25.584340 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:10:25.584340 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:10:25.584340 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 14:10:25.584340 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 14:10:25.584340 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 14:10:25.584340 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 14:10:25.584340 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 14:10:25.584340 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 14:10:25.584340 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 05 14:10:25.584340 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:10:25.584340 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 593/1212] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:10:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:10:25.588780 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f084f-e6ae-46d1-8efe-d840531932f6 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:25.687729 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f084f-e6ae-46d1-8efe-d840531932f6 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:25.721017 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6927de-a0be-4526-a095-0dffdce0fb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:25.721680 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6927de-a0be-4526-a095-0dffdce0fb66 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:25.721883 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6927de-a0be-4526-a095-0dffdce0fb66 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:25.722149 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6927de-a0be-4526-a095-0dffdce0fb66 None admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:25.722946 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6927de-a0be-4526-a095-0dffdce0fb66 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:25.723257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6927de-a0be-4526-a095-0dffdce0fb66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:25.724739 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6927de-a0be-4526-a095-0dffdce0fb66 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:25.742421 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 620/1213] 10.222.0.26 () {56 vars in 1143 bytes} [Fri Aug 5 14:10:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:25.748478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0fe424-fed5-4a8d-b100-dcbe18b47e2b None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:25.849443 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0fe424-fed5-4a8d-b100-dcbe18b47e2b None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:25.882531 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ee0d1-fb58-4523-92bd-f15e90db2736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:25.883130 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ee0d1-fb58-4523-92bd-f15e90db2736 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:25.883404 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ee0d1-fb58-4523-92bd-f15e90db2736 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:25.883669 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ee0d1-fb58-4523-92bd-f15e90db2736 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:25.913213 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:25.950750 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ee0d1-fb58-4523-92bd-f15e90db2736 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:25.952107 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ee0d1-fb58-4523-92bd-f15e90db2736 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:26.048215 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 594/1214] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:10:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 14:10:26.054266 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6927de-a0be-4526-a095-0dffdce0fb66 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:26.073105 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:26.151948 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6927de-a0be-4526-a095-0dffdce0fb66 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:26.185518 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052b7e8-522f-44b0-a3e9-3da6814fb0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:26.186154 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052b7e8-522f-44b0-a3e9-3da6814fb0df None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:26.186432 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052b7e8-522f-44b0-a3e9-3da6814fb0df None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:26.186705 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052b7e8-522f-44b0-a3e9-3da6814fb0df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:26.196536 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052b7e8-522f-44b0-a3e9-3da6814fb0df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:26.197980 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052b7e8-522f-44b0-a3e9-3da6814fb0df None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:26.213606 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-1052b7e8-522f-44b0-a3e9-3da6814fb0df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:10:26.213606 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:10:26.213606 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:10:26.213606 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:10:26.213606 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:10:26.213606 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:10:26.213606 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:10:26.213606 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:10:26.213606 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:10:26.213606 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:10:26.213606 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:10:26.213606 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:10:26.213606 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:10:26.213606 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:10:26.213606 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:10:26.214755 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:10:26.214755 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:10:26.214755 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:10:26.214755 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:10:26.214755 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:10:26.214755 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:10:26.214755 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:10:26.214755 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:10:26.214755 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:10:26.214755 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:10:26.214755 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:10:26.214755 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:10:26.214755 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:10:26.214755 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:10:26.214755 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:10:26.214755 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:10:26.214755 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:10:26.217082 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:10:26.217082 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:10:26.217082 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:10:26.217082 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:10:26.217082 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 621/1215] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:10:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:10:26.221187 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ee0d1-fb58-4523-92bd-f15e90db2736 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:26.280901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:26.317911 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ee0d1-fb58-4523-92bd-f15e90db2736 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:26.350780 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e69fa-de44-4871-a2d2-3dce5507157f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:26.351378 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e69fa-de44-4871-a2d2-3dce5507157f None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:26.351652 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e69fa-de44-4871-a2d2-3dce5507157f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:26.351917 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e69fa-de44-4871-a2d2-3dce5507157f None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:26.352617 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194e69fa-de44-4871-a2d2-3dce5507157f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:26.352914 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194e69fa-de44-4871-a2d2-3dce5507157f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:26.354246 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194e69fa-de44-4871-a2d2-3dce5507157f None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:26.365720 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 595/1216] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:10:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:26.371695 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052b7e8-522f-44b0-a3e9-3da6814fb0df None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:26.438736 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:26.469756 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052b7e8-522f-44b0-a3e9-3da6814fb0df None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:26.502735 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd4a14f-16a1-441d-bf70-a13f20c08392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:26.503377 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd4a14f-16a1-441d-bf70-a13f20c08392 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:26.503635 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd4a14f-16a1-441d-bf70-a13f20c08392 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:26.503906 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd4a14f-16a1-441d-bf70-a13f20c08392 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:26.510339 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd4a14f-16a1-441d-bf70-a13f20c08392 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:26.512642 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd4a14f-16a1-441d-bf70-a13f20c08392 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:26.524867 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-8bd4a14f-16a1-441d-bf70-a13f20c08392 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:10:26.524867 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:10:26.524867 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:10:26.524867 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:10:26.524867 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:10:26.524867 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:10:26.524867 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:10:26.524867 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:10:26.524867 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:10:26.524867 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:10:26.524867 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:10:26.524867 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:10:26.524867 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 14:10:26.524867 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 14:10:26.524867 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 14:10:26.525982 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 14:10:26.525982 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:10:26.525982 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:10:26.525982 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:10:26.525982 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:10:26.525982 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:10:26.525982 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:10:26.525982 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:10:26.525982 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:10:26.525982 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:10:26.525982 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:10:26.525982 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:10:26.525982 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:10:26.525982 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:10:26.525982 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:10:26.525982 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:10:26.525982 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:10:26.526901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 14:10:26.526901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 14:10:26.526901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:10:26.526901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:10:26.526901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:10:26.526901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:10:26.526901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:10:26.526901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:10:26.526901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 622/1217] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:10:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:10:26.533057 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e69fa-de44-4871-a2d2-3dce5507157f None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:26.630879 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e69fa-de44-4871-a2d2-3dce5507157f None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:26.655057 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:26.666315 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09ccc3-ee6c-4eab-9f8b-0705247ba1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:26.666944 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09ccc3-ee6c-4eab-9f8b-0705247ba1f2 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:26.667216 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09ccc3-ee6c-4eab-9f8b-0705247ba1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:26.667491 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09ccc3-ee6c-4eab-9f8b-0705247ba1f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:26.668297 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e09ccc3-ee6c-4eab-9f8b-0705247ba1f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:26.668641 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e09ccc3-ee6c-4eab-9f8b-0705247ba1f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:26.670071 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e09ccc3-ee6c-4eab-9f8b-0705247ba1f2 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:26.681294 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 596/1218] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Aug 5 14:10:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:26.688008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd4a14f-16a1-441d-bf70-a13f20c08392 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:26.787780 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd4a14f-16a1-441d-bf70-a13f20c08392 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:26.815449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:26.820850 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b809be2-9f2e-4db4-a597-969ad6f9f3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:26.821456 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b809be2-9f2e-4db4-a597-969ad6f9f3c0 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:26.821731 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b809be2-9f2e-4db4-a597-969ad6f9f3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:26.822002 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b809be2-9f2e-4db4-a597-969ad6f9f3c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:26.822780 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b809be2-9f2e-4db4-a597-969ad6f9f3c0 None admin] RBAC: Adding query filter params (role.id=ec18a2850fd540a39d4a935348bb4474, scope.project.id=319973e011c54b4ca1f2198c8a5e135b, user.id=fff761f6f81a46098f94bd2ec2be9c1a) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:26.823089 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b809be2-9f2e-4db4-a597-969ad6f9f3c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:26.825106 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b809be2-9f2e-4db4-a597-969ad6f9f3c0 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:26.830291 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 623/1219] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:10:26 2022] GET /identity/v3/role_assignments?user.id=fff761f6f81a46098f94bd2ec2be9c1a&scope.project.id=319973e011c54b4ca1f2198c8a5e135b&role.id=ec18a2850fd540a39d4a935348bb4474 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:28.827114 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b175f70-7dec-4bc8-8bb6-baad49e24c7d None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:28.827623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b175f70-7dec-4bc8-8bb6-baad49e24c7d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:28.827623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b175f70-7dec-4bc8-8bb6-baad49e24c7d None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:28.828940 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 597/1220] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:10:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:10:28.834155 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7c98b-b7f6-44f2-8fe1-dd15dcaa5ef7 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:28.834541 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7c98b-b7f6-44f2-8fe1-dd15dcaa5ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:28.834697 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7c98b-b7f6-44f2-8fe1-dd15dcaa5ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:28.854494 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-fcf7c98b-b7f6-44f2-8fe1-dd15dcaa5ef7 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:10:28.856373 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-fcf7c98b-b7f6-44f2-8fe1-dd15dcaa5ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:10:28.923114 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 624/1221] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:10:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:10:28.945513 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b175f70-7dec-4bc8-8bb6-baad49e24c7d None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:29.046077 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b175f70-7dec-4bc8-8bb6-baad49e24c7d None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:29.079212 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90bc65-da87-4cdb-a43e-9b12c0e596b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:29.079811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90bc65-da87-4cdb-a43e-9b12c0e596b9 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:29.080084 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90bc65-da87-4cdb-a43e-9b12c0e596b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:29.080350 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90bc65-da87-4cdb-a43e-9b12c0e596b9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:29.090491 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90bc65-da87-4cdb-a43e-9b12c0e596b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:10:29.095296 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-1b90bc65-da87-4cdb-a43e-9b12c0e596b9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:10:29.095296 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:10:29.095296 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:10:29.095296 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:10:29.095296 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:10:29.095296 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:10:29.095296 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:10:29.095296 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:10:29.095296 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:10:29.095296 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:10:29.095296 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:10:29.095296 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:10:29.095296 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:10:29.095296 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:10:29.095296 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:10:29.095296 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:10:29.101169 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:10:29.101169 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:10:29.101169 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:10:29.101169 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:10:29.101169 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:10:29.101169 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:10:29.101169 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:10:29.101169 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:10:29.101169 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:10:29.101169 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:10:29.101169 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:10:29.101169 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:10:29.101169 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:10:29.101169 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:10:29.101169 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:10:29.101169 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:10:29.102273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:10:29.102273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:10:29.102273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:10:29.102273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:10:29.102273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:10:29.102273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:10:29.102273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:10:29.102273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:10:29.102273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:10:29.102273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:10:29.102273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:10:29.102273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:10:29.102273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 598/1222] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Aug 5 14:10:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:10:29.103177 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf7c98b-b7f6-44f2-8fe1-dd15dcaa5ef7 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:29.201307 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf7c98b-b7f6-44f2-8fe1-dd15dcaa5ef7 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:29.224617 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:29.235102 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d266f4-d2be-435c-b8f9-2d46fa35990d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:29.235714 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d266f4-d2be-435c-b8f9-2d46fa35990d None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:29.235991 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d266f4-d2be-435c-b8f9-2d46fa35990d None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:29.236262 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d266f4-d2be-435c-b8f9-2d46fa35990d None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:29.237036 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d266f4-d2be-435c-b8f9-2d46fa35990d None admin] RBAC: Adding query filter params (name=service) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:29.237588 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d266f4-d2be-435c-b8f9-2d46fa35990d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:29.238943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d266f4-d2be-435c-b8f9-2d46fa35990d None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:29.246608 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 625/1223] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Aug 5 14:10:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:29.252457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90bc65-da87-4cdb-a43e-9b12c0e596b9 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:29.295930 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:29.350612 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90bc65-da87-4cdb-a43e-9b12c0e596b9 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:29.383045 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b1921-8a11-441f-a6ee-383d1c697186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:29.383676 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b1921-8a11-441f-a6ee-383d1c697186 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:29.383949 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b1921-8a11-441f-a6ee-383d1c697186 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:29.384218 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b1921-8a11-441f-a6ee-383d1c697186 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:29.394555 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5b1921-8a11-441f-a6ee-383d1c697186 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:29.395991 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5b1921-8a11-441f-a6ee-383d1c697186 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:29.410525 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-5e5b1921-8a11-441f-a6ee-383d1c697186 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:10:29.410525 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:10:29.410525 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:10:29.410525 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:10:29.410525 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:10:29.410525 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:10:29.410525 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:10:29.410525 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:10:29.410525 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:10:29.410525 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:10:29.410525 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:10:29.410525 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:10:29.410525 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:10:29.410525 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:10:29.410525 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:10:29.411634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:10:29.411634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:10:29.411634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:10:29.411634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:10:29.411634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:10:29.411634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:10:29.411634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:10:29.411634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:10:29.411634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:10:29.411634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:10:29.411634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:10:29.411634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:10:29.411634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:10:29.411634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:10:29.411634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:10:29.411634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:10:29.411634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:10:29.414401 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:10:29.414401 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:10:29.414401 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:10:29.414401 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:10:29.414401 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 599/1224] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:10:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:10:29.417618 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d266f4-d2be-435c-b8f9-2d46fa35990d None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:29.514670 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d266f4-d2be-435c-b8f9-2d46fa35990d None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:29.548579 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7065183-49f8-4b48-bcf9-44661f071798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:29.549188 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7065183-49f8-4b48-bcf9-44661f071798 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:29.549463 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7065183-49f8-4b48-bcf9-44661f071798 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:29.549735 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7065183-49f8-4b48-bcf9-44661f071798 None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:29.550407 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7065183-49f8-4b48-bcf9-44661f071798 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:29.550707 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7065183-49f8-4b48-bcf9-44661f071798 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:29.552056 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7065183-49f8-4b48-bcf9-44661f071798 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:29.564874 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 626/1225] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:10:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:29.570687 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b1921-8a11-441f-a6ee-383d1c697186 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:29.664230 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:29.670772 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b1921-8a11-441f-a6ee-383d1c697186 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:29.704186 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a969ddee-d540-474c-a788-13b9549ae8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:29.704859 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a969ddee-d540-474c-a788-13b9549ae8a3 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:29.705137 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a969ddee-d540-474c-a788-13b9549ae8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:29.705407 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a969ddee-d540-474c-a788-13b9549ae8a3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:29.712123 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a969ddee-d540-474c-a788-13b9549ae8a3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:29.713576 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a969ddee-d540-474c-a788-13b9549ae8a3 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:29.724979 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-a969ddee-d540-474c-a788-13b9549ae8a3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 05 14:10:29.724979 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:10:29.724979 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:10:29.724979 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:10:29.724979 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:10:29.724979 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:10:29.724979 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:10:29.724979 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:10:29.724979 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:10:29.724979 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:10:29.724979 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:10:29.724979 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:10:29.724979 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 14:10:29.724979 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 14:10:29.724979 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 14:10:29.724979 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 14:10:29.725946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:10:29.725946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:10:29.725946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:10:29.725946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:10:29.725946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:10:29.725946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:10:29.725946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:10:29.725946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:10:29.725946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:10:29.725946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:10:29.725946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:10:29.725946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:10:29.725946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:10:29.725946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:10:29.725946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:10:29.725946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:10:29.725946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:10:29.725946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:10:29.727007 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:10:29.727007 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:10:29.727007 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 14:10:29.727007 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 14:10:29.727007 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 14:10:29.727007 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 14:10:29.727007 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 14:10:29.727007 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 14:10:29.727007 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 05 14:10:29.727007 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:10:29.727007 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 600/1226] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:10:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:10:29.732486 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7065183-49f8-4b48-bcf9-44661f071798 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:29.778985 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:29.829901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7065183-49f8-4b48-bcf9-44661f071798 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:29.862817 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da1aed-c488-4859-9954-57dfd1043ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:29.863422 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da1aed-c488-4859-9954-57dfd1043ef8 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:29.863701 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da1aed-c488-4859-9954-57dfd1043ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:29.863973 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da1aed-c488-4859-9954-57dfd1043ef8 None admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:29.864785 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83da1aed-c488-4859-9954-57dfd1043ef8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:29.865091 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83da1aed-c488-4859-9954-57dfd1043ef8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:29.866507 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83da1aed-c488-4859-9954-57dfd1043ef8 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:29.883398 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 627/1227] 10.222.0.26 () {56 vars in 1143 bytes} [Fri Aug 5 14:10:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:29.889718 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a969ddee-d540-474c-a788-13b9549ae8a3 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:29.989781 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a969ddee-d540-474c-a788-13b9549ae8a3 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:30.022225 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24fb948-45a6-48c4-b96e-de39e3355e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:30.022843 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24fb948-45a6-48c4-b96e-de39e3355e48 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:30.023115 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24fb948-45a6-48c4-b96e-de39e3355e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:30.023377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24fb948-45a6-48c4-b96e-de39e3355e48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:30.034181 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24fb948-45a6-48c4-b96e-de39e3355e48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:30.035612 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24fb948-45a6-48c4-b96e-de39e3355e48 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:30.049392 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-e24fb948-45a6-48c4-b96e-de39e3355e48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:10:30.049392 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:10:30.049392 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:10:30.049392 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:10:30.049392 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:10:30.049392 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:10:30.049392 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:10:30.049392 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:10:30.049392 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:10:30.049392 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:10:30.049392 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:10:30.049392 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:10:30.049392 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:10:30.049392 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:10:30.049392 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:10:30.050441 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:10:30.050441 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:10:30.050441 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:10:30.050441 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:10:30.050441 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:10:30.050441 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:10:30.050441 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:10:30.050441 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:10:30.050441 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:10:30.050441 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:10:30.050441 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:10:30.050441 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:10:30.050441 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:10:30.050441 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:10:30.050441 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:10:30.050441 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:10:30.050441 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:10:30.051624 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:10:30.051624 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:10:30.051624 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:10:30.051624 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:10:30.051624 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 601/1228] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:10:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:10:30.056449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da1aed-c488-4859-9954-57dfd1043ef8 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:30.154756 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da1aed-c488-4859-9954-57dfd1043ef8 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:30.157046 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:30.186982 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6103838-0d0a-4755-9c9e-cd10b497c9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:30.187597 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6103838-0d0a-4755-9c9e-cd10b497c9eb None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:30.187925 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6103838-0d0a-4755-9c9e-cd10b497c9eb None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:30.188589 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6103838-0d0a-4755-9c9e-cd10b497c9eb None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:30.192654 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6103838-0d0a-4755-9c9e-cd10b497c9eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:30.192654 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6103838-0d0a-4755-9c9e-cd10b497c9eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:30.192654 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6103838-0d0a-4755-9c9e-cd10b497c9eb None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:30.203097 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 628/1229] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:10:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:30.208869 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24fb948-45a6-48c4-b96e-de39e3355e48 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:30.219524 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:30.304748 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24fb948-45a6-48c4-b96e-de39e3355e48 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:30.338161 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7159c0c-f35c-49b7-bfc4-998176a5605d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:30.338783 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7159c0c-f35c-49b7-bfc4-998176a5605d None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:30.339063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7159c0c-f35c-49b7-bfc4-998176a5605d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:30.339331 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7159c0c-f35c-49b7-bfc4-998176a5605d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:30.345936 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7159c0c-f35c-49b7-bfc4-998176a5605d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:30.347352 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7159c0c-f35c-49b7-bfc4-998176a5605d None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:30.358259 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-a7159c0c-f35c-49b7-bfc4-998176a5605d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:10:30.358259 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:10:30.358259 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:10:30.358259 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:10:30.358259 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:10:30.358259 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:10:30.358259 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:10:30.358259 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:10:30.358259 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:10:30.358259 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:10:30.358259 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:10:30.358259 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:10:30.358259 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 14:10:30.358259 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 14:10:30.358259 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 14:10:30.359331 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 14:10:30.359331 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:10:30.359331 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:10:30.359331 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:10:30.359331 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:10:30.359331 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:10:30.359331 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:10:30.359331 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:10:30.359331 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:10:30.359331 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:10:30.359331 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:10:30.359331 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:10:30.359331 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:10:30.359331 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:10:30.359331 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:10:30.359331 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:10:30.359331 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:10:30.370410 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 14:10:30.370410 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 14:10:30.370410 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:10:30.370410 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:10:30.370410 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:10:30.370410 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:10:30.370410 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:10:30.370410 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:10:30.370410 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 602/1230] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:10:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:10:30.371448 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6103838-0d0a-4755-9c9e-cd10b497c9eb None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:30.462895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6103838-0d0a-4755-9c9e-cd10b497c9eb None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:30.495751 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec8afc-b013-4690-902e-92f51dd12eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:30.496915 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec8afc-b013-4690-902e-92f51dd12eae None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:30.497189 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec8afc-b013-4690-902e-92f51dd12eae None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:30.497454 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec8afc-b013-4690-902e-92f51dd12eae None admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:30.498191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec8afc-b013-4690-902e-92f51dd12eae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:30.498492 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec8afc-b013-4690-902e-92f51dd12eae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:30.499956 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec8afc-b013-4690-902e-92f51dd12eae None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:30.512146 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 629/1231] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Aug 5 14:10:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:30.518086 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7159c0c-f35c-49b7-bfc4-998176a5605d None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:30.583989 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:30.617038 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7159c0c-f35c-49b7-bfc4-998176a5605d None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:30.650457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0479a24f-8e17-4184-8220-38264f731bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:30.651112 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479a24f-8e17-4184-8220-38264f731bda None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:30.651388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479a24f-8e17-4184-8220-38264f731bda None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:30.651658 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479a24f-8e17-4184-8220-38264f731bda None admin] PATH_INFO: `/v3/projects/319973e011c54b4ca1f2198c8a5e135b/users/fff761f6f81a46098f94bd2ec2be9c1a/roles/ec18a2850fd540a39d4a935348bb4474` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:30.665411 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0479a24f-8e17-4184-8220-38264f731bda None admin] RBAC: Authorizing `identity:create_grant(project_id=319973e011c54b4ca1f2198c8a5e135b, user_id=fff761f6f81a46098f94bd2ec2be9c1a, role_id=ec18a2850fd540a39d4a935348bb4474)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:30.666838 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0479a24f-8e17-4184-8220-38264f731bda None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:30.763149 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 603/1232] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Aug 5 14:10:30 2022] PUT /identity/v3/projects/319973e011c54b4ca1f2198c8a5e135b/users/fff761f6f81a46098f94bd2ec2be9c1a/roles/ec18a2850fd540a39d4a935348bb4474 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:10:32.750861 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ec066-d281-423a-9e6b-3d6dfa273531 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:32.751434 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ec066-d281-423a-9e6b-3d6dfa273531 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:32.751434 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ec066-d281-423a-9e6b-3d6dfa273531 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:32.752800 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 630/1233] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:10:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:10:32.757856 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd0787-c447-4777-a227-c06fac051e0e None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:32.758281 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd0787-c447-4777-a227-c06fac051e0e None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:32.758422 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd0787-c447-4777-a227-c06fac051e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:32.777165 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-27cd0787-c447-4777-a227-c06fac051e0e None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:10:32.778312 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-27cd0787-c447-4777-a227-c06fac051e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:10:32.843064 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 604/1234] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:10:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:10:32.866699 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ec066-d281-423a-9e6b-3d6dfa273531 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:32.892187 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:32.967713 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ec066-d281-423a-9e6b-3d6dfa273531 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:33.000752 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4667084b-cc5d-48d2-9f0d-66ffcf38a52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:33.001383 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667084b-cc5d-48d2-9f0d-66ffcf38a52b None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:33.001663 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667084b-cc5d-48d2-9f0d-66ffcf38a52b None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:33.001941 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667084b-cc5d-48d2-9f0d-66ffcf38a52b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:33.012394 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4667084b-cc5d-48d2-9f0d-66ffcf38a52b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:10:33.017411 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-4667084b-cc5d-48d2-9f0d-66ffcf38a52b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:10:33.017411 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:10:33.017411 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:10:33.017411 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:10:33.017411 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:10:33.017411 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:10:33.017411 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:10:33.017411 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:10:33.017411 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:10:33.017411 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:10:33.017411 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:10:33.017411 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:10:33.017411 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:10:33.017411 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:10:33.017411 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:10:33.017411 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:10:33.018497 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:10:33.018497 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:10:33.018497 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:10:33.018497 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:10:33.018497 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:10:33.018497 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:10:33.018497 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:10:33.018497 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:10:33.018497 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:10:33.018497 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:10:33.018497 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:10:33.018497 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:10:33.018497 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:10:33.018497 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:10:33.018497 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:10:33.018497 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:10:33.019667 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:10:33.019667 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:10:33.019667 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:10:33.019667 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:10:33.019667 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:10:33.019667 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:10:33.019667 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:10:33.019667 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:10:33.019667 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:10:33.019667 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:10:33.019667 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 14:10:33.019667 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:10:33.019667 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 631/1235] 10.222.0.26 () {56 vars in 1102 bytes} [Fri Aug 5 14:10:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:10:33.024763 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cd0787-c447-4777-a227-c06fac051e0e None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:33.070437 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:33.122683 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cd0787-c447-4777-a227-c06fac051e0e None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:33.157091 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37cc407-b64b-4cd1-bad4-75f8769b6bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:33.157706 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37cc407-b64b-4cd1-bad4-75f8769b6bdb None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:33.157985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37cc407-b64b-4cd1-bad4-75f8769b6bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:33.158257 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37cc407-b64b-4cd1-bad4-75f8769b6bdb None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:33.158961 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37cc407-b64b-4cd1-bad4-75f8769b6bdb None admin] RBAC: Adding query filter params (name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:33.159263 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37cc407-b64b-4cd1-bad4-75f8769b6bdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:33.160641 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37cc407-b64b-4cd1-bad4-75f8769b6bdb None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:33.168945 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 605/1236] 10.222.0.26 () {56 vars in 1103 bytes} [Fri Aug 5 14:10:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:33.174813 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4667084b-cc5d-48d2-9f0d-66ffcf38a52b None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:33.257995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:33.274109 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4667084b-cc5d-48d2-9f0d-66ffcf38a52b None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:33.306806 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20747e0b-a806-44ac-aea5-7a7dfd13c818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:33.307432 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20747e0b-a806-44ac-aea5-7a7dfd13c818 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:33.307709 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20747e0b-a806-44ac-aea5-7a7dfd13c818 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:33.307978 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20747e0b-a806-44ac-aea5-7a7dfd13c818 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:33.318593 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20747e0b-a806-44ac-aea5-7a7dfd13c818 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:33.320019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20747e0b-a806-44ac-aea5-7a7dfd13c818 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:33.335052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-20747e0b-a806-44ac-aea5-7a7dfd13c818 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:10:33.335052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:10:33.335052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:10:33.335052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:10:33.335052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:10:33.335052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:10:33.335052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:10:33.335052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:10:33.335052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:10:33.335052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:10:33.335052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:10:33.335052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:10:33.335052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:10:33.335052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:10:33.335052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:10:33.336106 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:10:33.336106 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:10:33.336106 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:10:33.336106 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:10:33.336106 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:10:33.336106 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:10:33.336106 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:10:33.336106 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:10:33.336106 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:10:33.336106 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:10:33.336106 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:10:33.336106 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:10:33.336106 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:10:33.336106 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:10:33.336106 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:10:33.336106 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:10:33.336106 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:10:33.337270 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:10:33.337270 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:10:33.337270 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:10:33.337270 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:10:33.337270 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 632/1237] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:10:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:10:33.342311 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37cc407-b64b-4cd1-bad4-75f8769b6bdb None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:33.439811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37cc407-b64b-4cd1-bad4-75f8769b6bdb None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:33.448596 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:33.472506 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49bd5b-b634-4151-9e22-93130fb0fa24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:33.473109 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49bd5b-b634-4151-9e22-93130fb0fa24 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:33.473384 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49bd5b-b634-4151-9e22-93130fb0fa24 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:33.473652 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49bd5b-b634-4151-9e22-93130fb0fa24 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:33.474333 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed49bd5b-b634-4151-9e22-93130fb0fa24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:33.474627 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed49bd5b-b634-4151-9e22-93130fb0fa24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:33.476233 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed49bd5b-b634-4151-9e22-93130fb0fa24 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:33.487956 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 606/1238] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:10:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:33.493637 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20747e0b-a806-44ac-aea5-7a7dfd13c818 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:33.592116 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20747e0b-a806-44ac-aea5-7a7dfd13c818 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:33.625253 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0bba8c-b28a-48d0-8a10-1351a922cfc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:33.625881 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0bba8c-b28a-48d0-8a10-1351a922cfc3 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:33.626158 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0bba8c-b28a-48d0-8a10-1351a922cfc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:33.626429 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0bba8c-b28a-48d0-8a10-1351a922cfc3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:33.629361 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:33.634779 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0bba8c-b28a-48d0-8a10-1351a922cfc3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:33.636204 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0bba8c-b28a-48d0-8a10-1351a922cfc3 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:33.648308 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-ff0bba8c-b28a-48d0-8a10-1351a922cfc3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 05 14:10:33.648308 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:10:33.648308 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:10:33.648308 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:10:33.648308 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:10:33.648308 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:10:33.648308 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:10:33.648308 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:10:33.648308 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:10:33.648308 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:10:33.648308 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:10:33.648308 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:10:33.648308 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 14:10:33.648308 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 14:10:33.648308 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 14:10:33.648308 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 14:10:33.649375 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:10:33.649375 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:10:33.649375 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:10:33.649375 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:10:33.649375 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:10:33.649375 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:10:33.649375 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:10:33.649375 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:10:33.649375 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:10:33.649375 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:10:33.649375 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:10:33.649375 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:10:33.649375 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:10:33.649375 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:10:33.649375 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:10:33.649375 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:10:33.649375 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:10:33.649375 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:10:33.650791 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:10:33.650791 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:10:33.650791 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 14:10:33.650791 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 14:10:33.650791 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 14:10:33.650791 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 14:10:33.650791 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 14:10:33.650791 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 14:10:33.650791 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 05 14:10:33.650791 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:10:33.650791 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 633/1239] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:10:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:10:33.656023 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49bd5b-b634-4151-9e22-93130fb0fa24 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:33.754742 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49bd5b-b634-4151-9e22-93130fb0fa24 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:33.788637 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd66484-9bdd-47a0-8a25-112a65cf7f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:33.789252 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd66484-9bdd-47a0-8a25-112a65cf7f05 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:33.789500 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd66484-9bdd-47a0-8a25-112a65cf7f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:33.789762 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd66484-9bdd-47a0-8a25-112a65cf7f05 None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:33.790580 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd66484-9bdd-47a0-8a25-112a65cf7f05 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:33.790879 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd66484-9bdd-47a0-8a25-112a65cf7f05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:33.792277 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd66484-9bdd-47a0-8a25-112a65cf7f05 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:33.809123 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 607/1240] 10.222.0.26 () {56 vars in 1143 bytes} [Fri Aug 5 14:10:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:33.815163 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0bba8c-b28a-48d0-8a10-1351a922cfc3 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:33.915040 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0bba8c-b28a-48d0-8a10-1351a922cfc3 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:33.947384 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc30d1-7b66-4215-b6ab-ed94ff16b94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:33.947996 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc30d1-7b66-4215-b6ab-ed94ff16b94a None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:33.948276 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc30d1-7b66-4215-b6ab-ed94ff16b94a None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:33.948578 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc30d1-7b66-4215-b6ab-ed94ff16b94a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:34.007476 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:34.014353 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc30d1-7b66-4215-b6ab-ed94ff16b94a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:34.015790 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc30d1-7b66-4215-b6ab-ed94ff16b94a None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:34.110796 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 634/1241] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:10:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 14:10:34.116953 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd66484-9bdd-47a0-8a25-112a65cf7f05 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:34.144610 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:34.214091 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd66484-9bdd-47a0-8a25-112a65cf7f05 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:34.248298 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb872a-a1e6-4c87-a06d-7045e1322493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:34.249849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb872a-a1e6-4c87-a06d-7045e1322493 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:34.250116 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb872a-a1e6-4c87-a06d-7045e1322493 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:34.250390 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb872a-a1e6-4c87-a06d-7045e1322493 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:34.261052 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eb872a-a1e6-4c87-a06d-7045e1322493 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:34.263250 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eb872a-a1e6-4c87-a06d-7045e1322493 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:34.277544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-a2eb872a-a1e6-4c87-a06d-7045e1322493 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:10:34.277544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:10:34.277544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:10:34.277544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:10:34.277544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:10:34.277544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:10:34.277544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:10:34.277544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:10:34.277544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:10:34.277544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:10:34.277544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:10:34.277544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:10:34.277544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:10:34.277544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:10:34.277544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:10:34.278603 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:10:34.278603 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:10:34.278603 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:10:34.278603 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:10:34.278603 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:10:34.278603 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:10:34.278603 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:10:34.278603 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:10:34.278603 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:10:34.278603 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:10:34.278603 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:10:34.278603 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:10:34.278603 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:10:34.278603 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:10:34.278603 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:10:34.278603 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:10:34.278603 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:10:34.286062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:10:34.286062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:10:34.286062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:10:34.286062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:10:34.286062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 608/1242] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:10:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:10:34.286436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc30d1-7b66-4215-b6ab-ed94ff16b94a None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:34.376790 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:34.382412 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc30d1-7b66-4215-b6ab-ed94ff16b94a None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:34.415749 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d91432-e233-433c-badb-efd8d07422a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:34.416354 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d91432-e233-433c-badb-efd8d07422a7 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:34.416664 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d91432-e233-433c-badb-efd8d07422a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:34.416937 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d91432-e233-433c-badb-efd8d07422a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:34.418274 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d91432-e233-433c-badb-efd8d07422a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:34.418632 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d91432-e233-433c-badb-efd8d07422a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:34.420000 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d91432-e233-433c-badb-efd8d07422a7 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:34.430896 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 635/1243] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:10:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:34.437152 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb872a-a1e6-4c87-a06d-7045e1322493 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:34.513731 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:34.535873 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb872a-a1e6-4c87-a06d-7045e1322493 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:34.569861 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff47ecd6-239d-4f78-bd3e-b4d4eec6200a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:34.570496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47ecd6-239d-4f78-bd3e-b4d4eec6200a None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:34.570827 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47ecd6-239d-4f78-bd3e-b4d4eec6200a None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:34.571098 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47ecd6-239d-4f78-bd3e-b4d4eec6200a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:34.578197 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff47ecd6-239d-4f78-bd3e-b4d4eec6200a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:34.579621 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff47ecd6-239d-4f78-bd3e-b4d4eec6200a None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:34.591808 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-ff47ecd6-239d-4f78-bd3e-b4d4eec6200a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:10:34.591808 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:10:34.591808 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:10:34.591808 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:10:34.591808 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:10:34.591808 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:10:34.591808 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:10:34.591808 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:10:34.591808 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:10:34.591808 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:10:34.591808 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:10:34.591808 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:10:34.591808 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 14:10:34.591808 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 14:10:34.591808 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 14:10:34.592906 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 14:10:34.592906 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:10:34.592906 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:10:34.592906 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:10:34.592906 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:10:34.592906 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:10:34.592906 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:10:34.592906 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:10:34.592906 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:10:34.592906 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:10:34.592906 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:10:34.592906 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:10:34.592906 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:10:34.592906 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:10:34.592906 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:10:34.592906 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:10:34.592906 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:10:34.594048 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 14:10:34.594048 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 14:10:34.594048 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:10:34.594048 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:10:34.594048 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:10:34.594048 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:10:34.594048 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:10:34.594048 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:10:34.594048 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 609/1244] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:10:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:10:34.599058 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d91432-e233-433c-badb-efd8d07422a7 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:34.698838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d91432-e233-433c-badb-efd8d07422a7 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:34.733625 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b2328-81ba-4fe6-9426-99ad544ff81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:34.734232 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b2328-81ba-4fe6-9426-99ad544ff81e None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:34.734511 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b2328-81ba-4fe6-9426-99ad544ff81e None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:34.734783 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b2328-81ba-4fe6-9426-99ad544ff81e None admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:34.735525 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02b2328-81ba-4fe6-9426-99ad544ff81e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:34.735835 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02b2328-81ba-4fe6-9426-99ad544ff81e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:34.737533 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02b2328-81ba-4fe6-9426-99ad544ff81e None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:34.749774 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 636/1245] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Aug 5 14:10:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:34.756118 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff47ecd6-239d-4f78-bd3e-b4d4eec6200a None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:34.858099 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff47ecd6-239d-4f78-bd3e-b4d4eec6200a None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:34.890486 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af56ab13-1467-4c6c-b9f7-40627870b01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:34.891096 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56ab13-1467-4c6c-b9f7-40627870b01b None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:34.891377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56ab13-1467-4c6c-b9f7-40627870b01b None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:34.891666 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56ab13-1467-4c6c-b9f7-40627870b01b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:34.892443 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af56ab13-1467-4c6c-b9f7-40627870b01b None admin] RBAC: Adding query filter params (role.id=ec18a2850fd540a39d4a935348bb4474, scope.project.id=319973e011c54b4ca1f2198c8a5e135b, user.id=fff761f6f81a46098f94bd2ec2be9c1a) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:34.892754 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af56ab13-1467-4c6c-b9f7-40627870b01b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:34.894163 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af56ab13-1467-4c6c-b9f7-40627870b01b None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:34.899818 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 610/1246] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:10:34 2022] GET /identity/v3/role_assignments?user.id=fff761f6f81a46098f94bd2ec2be9c1a&scope.project.id=319973e011c54b4ca1f2198c8a5e135b&role.id=ec18a2850fd540a39d4a935348bb4474 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:36.937606 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be042f79-426b-42a3-a0b0-c0bad7c3c272 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:36.938162 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be042f79-426b-42a3-a0b0-c0bad7c3c272 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:36.938162 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be042f79-426b-42a3-a0b0-c0bad7c3c272 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:36.939460 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 637/1247] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:10:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:10:36.944567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a6d60-488e-4160-a0b4-860dee6abba0 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:36.945147 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a6d60-488e-4160-a0b4-860dee6abba0 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:36.945147 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a6d60-488e-4160-a0b4-860dee6abba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:36.951957 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:36.963780 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-807a6d60-488e-4160-a0b4-860dee6abba0 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:10:36.965593 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-807a6d60-488e-4160-a0b4-860dee6abba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:10:37.031041 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 611/1248] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:10:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:10:37.053863 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be042f79-426b-42a3-a0b0-c0bad7c3c272 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:37.072103 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:37.153926 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be042f79-426b-42a3-a0b0-c0bad7c3c272 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:37.187208 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcede09-c841-4be6-97ff-3c48b73440d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:37.187884 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcede09-c841-4be6-97ff-3c48b73440d0 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:37.188108 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcede09-c841-4be6-97ff-3c48b73440d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:37.188373 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcede09-c841-4be6-97ff-3c48b73440d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:37.199379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcede09-c841-4be6-97ff-3c48b73440d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:10:37.204261 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-8fcede09-c841-4be6-97ff-3c48b73440d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:10:37.204261 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:10:37.204261 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:10:37.204261 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:10:37.204261 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:10:37.204261 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:10:37.204261 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:10:37.204261 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:10:37.204261 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:10:37.204261 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:10:37.204261 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:10:37.204261 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:10:37.204261 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:10:37.204261 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:10:37.204261 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:10:37.204261 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:10:37.205378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:10:37.205378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:10:37.205378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:10:37.205378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:10:37.205378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:10:37.205378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:10:37.205378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:10:37.205378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:10:37.205378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:10:37.205378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:10:37.205378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:10:37.205378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:10:37.205378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:10:37.205378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:10:37.205378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:10:37.205378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:10:37.206547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:10:37.206547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:10:37.206547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:10:37.206547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:10:37.206547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:10:37.206547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:10:37.206547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:10:37.206547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:10:37.206547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:10:37.206547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:10:37.206547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:10:37.206547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:10:37.206547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 638/1249] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Aug 5 14:10:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:10:37.211447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a6d60-488e-4160-a0b4-860dee6abba0 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:37.308567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a6d60-488e-4160-a0b4-860dee6abba0 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:37.341001 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:37.345745 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718487e8-f4a5-4f52-8855-a23b8fbde43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:37.346356 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718487e8-f4a5-4f52-8855-a23b8fbde43e None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:37.346634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718487e8-f4a5-4f52-8855-a23b8fbde43e None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:37.346906 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718487e8-f4a5-4f52-8855-a23b8fbde43e None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:37.347659 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718487e8-f4a5-4f52-8855-a23b8fbde43e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:37.347956 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718487e8-f4a5-4f52-8855-a23b8fbde43e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:37.349331 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718487e8-f4a5-4f52-8855-a23b8fbde43e None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:37.357365 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 612/1250] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:10:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:37.363507 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcede09-c841-4be6-97ff-3c48b73440d0 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:37.441494 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:37.463484 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcede09-c841-4be6-97ff-3c48b73440d0 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:37.495881 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90c3b9-0579-48f6-b590-a5a1af54c6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:37.496578 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90c3b9-0579-48f6-b590-a5a1af54c6dd None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:37.496856 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90c3b9-0579-48f6-b590-a5a1af54c6dd None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:37.497126 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90c3b9-0579-48f6-b590-a5a1af54c6dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:37.506683 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca90c3b9-0579-48f6-b590-a5a1af54c6dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:37.508138 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca90c3b9-0579-48f6-b590-a5a1af54c6dd None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:37.522882 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-ca90c3b9-0579-48f6-b590-a5a1af54c6dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:10:37.522882 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:10:37.522882 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:10:37.522882 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:10:37.522882 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:10:37.522882 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:10:37.522882 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:10:37.522882 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:10:37.522882 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:10:37.522882 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:10:37.522882 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:10:37.522882 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:10:37.522882 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:10:37.522882 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:10:37.522882 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:10:37.523974 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:10:37.523974 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:10:37.523974 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:10:37.523974 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:10:37.523974 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:10:37.523974 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:10:37.523974 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:10:37.523974 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:10:37.523974 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:10:37.523974 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:10:37.523974 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:10:37.523974 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:10:37.523974 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:10:37.523974 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:10:37.523974 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:10:37.523974 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:10:37.523974 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:10:37.525165 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:10:37.525165 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:10:37.525165 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:10:37.525165 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:10:37.525165 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 639/1251] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:10:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:10:37.530609 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718487e8-f4a5-4f52-8855-a23b8fbde43e None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:37.642072 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718487e8-f4a5-4f52-8855-a23b8fbde43e None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:37.675996 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9dcb26-d3cf-4f9d-8996-5945ebdd865d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:37.680310 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9dcb26-d3cf-4f9d-8996-5945ebdd865d None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:37.680310 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9dcb26-d3cf-4f9d-8996-5945ebdd865d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:37.680310 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9dcb26-d3cf-4f9d-8996-5945ebdd865d None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:37.680310 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9dcb26-d3cf-4f9d-8996-5945ebdd865d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:37.680310 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9dcb26-d3cf-4f9d-8996-5945ebdd865d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:37.680310 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9dcb26-d3cf-4f9d-8996-5945ebdd865d None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:37.691046 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 613/1252] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:10:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:37.696850 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90c3b9-0579-48f6-b590-a5a1af54c6dd None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:37.796250 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90c3b9-0579-48f6-b590-a5a1af54c6dd None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:37.821802 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:37.830101 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32be1b53-798c-4eeb-95ef-199b62446c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:37.830730 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be1b53-798c-4eeb-95ef-199b62446c35 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:37.831115 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be1b53-798c-4eeb-95ef-199b62446c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:37.831388 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be1b53-798c-4eeb-95ef-199b62446c35 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:37.839399 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32be1b53-798c-4eeb-95ef-199b62446c35 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:37.841957 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32be1b53-798c-4eeb-95ef-199b62446c35 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:37.851774 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-32be1b53-798c-4eeb-95ef-199b62446c35 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 05 14:10:37.851774 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:10:37.851774 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:10:37.851774 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:10:37.851774 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:10:37.851774 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:10:37.851774 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:10:37.851774 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:10:37.851774 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:10:37.851774 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:10:37.851774 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:10:37.851774 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:10:37.851774 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 14:10:37.851774 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 14:10:37.851774 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 14:10:37.851774 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 14:10:37.852873 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:10:37.852873 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:10:37.852873 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:10:37.852873 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:10:37.852873 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:10:37.852873 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:10:37.852873 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:10:37.852873 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:10:37.852873 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:10:37.852873 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:10:37.852873 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:10:37.852873 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:10:37.852873 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:10:37.852873 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:10:37.852873 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:10:37.852873 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:10:37.852873 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:10:37.852873 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:10:37.854113 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:10:37.854113 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:10:37.854113 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 14:10:37.854113 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 14:10:37.854113 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 14:10:37.854113 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 14:10:37.854113 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 14:10:37.854113 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 14:10:37.854113 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 05 14:10:37.854113 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:10:37.854113 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 640/1253] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:10:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:10:37.859176 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9dcb26-d3cf-4f9d-8996-5945ebdd865d None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:37.914191 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:37.958323 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9dcb26-d3cf-4f9d-8996-5945ebdd865d None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:37.991396 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321e6d1-6b93-4cfa-8331-210808af532d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:37.992036 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321e6d1-6b93-4cfa-8331-210808af532d None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:37.992319 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321e6d1-6b93-4cfa-8331-210808af532d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:37.992623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321e6d1-6b93-4cfa-8331-210808af532d None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:37.993416 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6321e6d1-6b93-4cfa-8331-210808af532d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:37.993721 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6321e6d1-6b93-4cfa-8331-210808af532d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:37.995142 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6321e6d1-6b93-4cfa-8331-210808af532d None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:38.013044 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 614/1254] 10.222.0.26 () {56 vars in 1143 bytes} [Fri Aug 5 14:10:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:38.018857 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32be1b53-798c-4eeb-95ef-199b62446c35 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:38.115945 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32be1b53-798c-4eeb-95ef-199b62446c35 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:38.152047 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69757bd6-be9a-43a8-a7f5-478663ed0ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:38.152578 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69757bd6-be9a-43a8-a7f5-478663ed0ce6 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:38.152855 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69757bd6-be9a-43a8-a7f5-478663ed0ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:38.153122 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69757bd6-be9a-43a8-a7f5-478663ed0ce6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:38.206577 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:38.218420 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69757bd6-be9a-43a8-a7f5-478663ed0ce6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:38.219778 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69757bd6-be9a-43a8-a7f5-478663ed0ce6 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:38.314257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 641/1255] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:10:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 14:10:38.320105 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321e6d1-6b93-4cfa-8331-210808af532d None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:38.419291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321e6d1-6b93-4cfa-8331-210808af532d None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:38.433892 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:38.454346 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f5f23-5270-4b78-9972-76122fba75fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:38.454980 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f5f23-5270-4b78-9972-76122fba75fb None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:38.455259 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f5f23-5270-4b78-9972-76122fba75fb None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:38.455530 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f5f23-5270-4b78-9972-76122fba75fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:38.465433 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846f5f23-5270-4b78-9972-76122fba75fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:38.466856 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846f5f23-5270-4b78-9972-76122fba75fb None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:38.481828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-846f5f23-5270-4b78-9972-76122fba75fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:10:38.481828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:10:38.481828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:10:38.481828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:10:38.481828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:10:38.481828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:10:38.481828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:10:38.481828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:10:38.481828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:10:38.481828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:10:38.481828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:10:38.481828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:10:38.481828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:10:38.481828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:10:38.481828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:10:38.482946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:10:38.482946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:10:38.482946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:10:38.482946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:10:38.482946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:10:38.482946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:10:38.482946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:10:38.482946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:10:38.482946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:10:38.482946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:10:38.482946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:10:38.482946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:10:38.482946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:10:38.482946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:10:38.482946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:10:38.482946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:10:38.482946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:10:38.484119 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:10:38.484119 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:10:38.484119 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:10:38.484119 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:10:38.484119 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 615/1256] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:10:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:10:38.494192 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69757bd6-be9a-43a8-a7f5-478663ed0ce6 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:38.575216 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:38.586646 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69757bd6-be9a-43a8-a7f5-478663ed0ce6 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:38.620841 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43010197-66f6-4799-8722-5e07f3e6295c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:38.621448 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43010197-66f6-4799-8722-5e07f3e6295c None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:38.621728 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43010197-66f6-4799-8722-5e07f3e6295c None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:38.622001 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43010197-66f6-4799-8722-5e07f3e6295c None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:38.622690 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43010197-66f6-4799-8722-5e07f3e6295c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:38.622994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43010197-66f6-4799-8722-5e07f3e6295c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:38.624341 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43010197-66f6-4799-8722-5e07f3e6295c None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:38.635317 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 642/1257] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:10:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:38.641280 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f5f23-5270-4b78-9972-76122fba75fb None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:38.739955 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f5f23-5270-4b78-9972-76122fba75fb None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:38.772472 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f3fb0a-7d44-4079-913f-9866264c6fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:38.773098 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3fb0a-7d44-4079-913f-9866264c6fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:38.773376 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3fb0a-7d44-4079-913f-9866264c6fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:38.773646 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3fb0a-7d44-4079-913f-9866264c6fc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:38.780731 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f3fb0a-7d44-4079-913f-9866264c6fc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:38.782150 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f3fb0a-7d44-4079-913f-9866264c6fc4 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:38.793888 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-73f3fb0a-7d44-4079-913f-9866264c6fc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:10:38.793888 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:10:38.793888 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:10:38.793888 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:10:38.793888 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:10:38.793888 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:10:38.793888 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:10:38.793888 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:10:38.793888 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:10:38.793888 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:10:38.793888 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:10:38.793888 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:10:38.793888 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 14:10:38.793888 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 14:10:38.793888 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 14:10:38.794970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 14:10:38.794970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:10:38.794970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:10:38.794970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:10:38.794970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:10:38.794970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:10:38.794970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:10:38.794970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:10:38.794970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:10:38.794970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:10:38.794970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:10:38.794970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:10:38.794970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:10:38.794970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:10:38.794970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:10:38.794970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:10:38.794970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:10:38.796149 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 14:10:38.796149 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 14:10:38.796149 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:10:38.796149 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:10:38.796149 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:10:38.796149 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:10:38.796149 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:10:38.796149 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:10:38.796149 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 616/1258] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:10:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:10:38.801889 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43010197-66f6-4799-8722-5e07f3e6295c None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:38.900595 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43010197-66f6-4799-8722-5e07f3e6295c None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:38.934203 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf59cb64-3d45-442c-9f11-46dcfd19c55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:38.934806 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59cb64-3d45-442c-9f11-46dcfd19c55e None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:38.935085 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59cb64-3d45-442c-9f11-46dcfd19c55e None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:38.935358 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59cb64-3d45-442c-9f11-46dcfd19c55e None admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:38.936098 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf59cb64-3d45-442c-9f11-46dcfd19c55e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:38.936509 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf59cb64-3d45-442c-9f11-46dcfd19c55e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:38.937920 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf59cb64-3d45-442c-9f11-46dcfd19c55e None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:38.949673 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 643/1259] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Aug 5 14:10:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:38.955817 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f3fb0a-7d44-4079-913f-9866264c6fc4 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:38.981401 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:39.051789 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f3fb0a-7d44-4079-913f-9866264c6fc4 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:39.086336 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a8718d-85a4-4a6b-ad1b-80627177f6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:39.086945 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8718d-85a4-4a6b-ad1b-80627177f6c6 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:39.087232 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8718d-85a4-4a6b-ad1b-80627177f6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:39.087503 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8718d-85a4-4a6b-ad1b-80627177f6c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:39.088277 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a8718d-85a4-4a6b-ad1b-80627177f6c6 None admin] RBAC: Adding query filter params (role.id=299c794c7acb4a5ca36288c3191aad10, scope.project.id=319973e011c54b4ca1f2198c8a5e135b, user.id=fff761f6f81a46098f94bd2ec2be9c1a) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:39.088626 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a8718d-85a4-4a6b-ad1b-80627177f6c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:39.090226 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a8718d-85a4-4a6b-ad1b-80627177f6c6 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:39.095472 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 617/1260] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:10:38 2022] GET /identity/v3/role_assignments?user.id=fff761f6f81a46098f94bd2ec2be9c1a&scope.project.id=319973e011c54b4ca1f2198c8a5e135b&role.id=299c794c7acb4a5ca36288c3191aad10 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:41.064365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be1bd6-c069-4da7-a730-05ceab6f9dbe None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:41.064996 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be1bd6-c069-4da7-a730-05ceab6f9dbe None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:41.064996 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be1bd6-c069-4da7-a730-05ceab6f9dbe None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:41.066288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 644/1261] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:10:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:10:41.071530 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3fbf3-77a5-4d22-83f5-39a35e923950 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:41.072134 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3fbf3-77a5-4d22-83f5-39a35e923950 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:41.072134 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3fbf3-77a5-4d22-83f5-39a35e923950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:41.090293 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-4dc3fbf3-77a5-4d22-83f5-39a35e923950 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:10:41.091546 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-4dc3fbf3-77a5-4d22-83f5-39a35e923950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:10:41.156877 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 618/1262] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:10:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:10:41.180211 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be1bd6-c069-4da7-a730-05ceab6f9dbe None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:41.191083 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:41.281220 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be1bd6-c069-4da7-a730-05ceab6f9dbe None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:41.314734 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a178fcc-5a98-4da2-9a94-1d6dc627fa7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:41.315295 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a178fcc-5a98-4da2-9a94-1d6dc627fa7a None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:41.315568 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a178fcc-5a98-4da2-9a94-1d6dc627fa7a None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:41.315835 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a178fcc-5a98-4da2-9a94-1d6dc627fa7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:41.326732 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a178fcc-5a98-4da2-9a94-1d6dc627fa7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:10:41.331548 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-1a178fcc-5a98-4da2-9a94-1d6dc627fa7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:10:41.331548 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:10:41.331548 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:10:41.331548 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:10:41.331548 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:10:41.331548 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:10:41.331548 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:10:41.331548 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:10:41.331548 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:10:41.331548 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:10:41.331548 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:10:41.331548 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:10:41.331548 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:10:41.331548 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:10:41.331548 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:10:41.331548 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:10:41.334960 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:10:41.334960 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:10:41.334960 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:10:41.334960 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:10:41.334960 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:10:41.334960 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:10:41.334960 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:10:41.334960 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:10:41.334960 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:10:41.334960 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:10:41.334960 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:10:41.334960 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:10:41.334960 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:10:41.334960 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:10:41.334960 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:10:41.334960 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:10:41.335817 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:10:41.335817 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:10:41.335817 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:10:41.335817 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:10:41.335817 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:10:41.335817 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:10:41.335817 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:10:41.335817 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:10:41.335817 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:10:41.335817 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:10:41.335817 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:10:41.335817 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:10:41.335817 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 645/1263] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Aug 5 14:10:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:10:41.338825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc3fbf3-77a5-4d22-83f5-39a35e923950 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:41.357295 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:41.434476 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc3fbf3-77a5-4d22-83f5-39a35e923950 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:41.468039 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3839e58-18a9-4f1d-a85a-e06f1f3348ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:41.469327 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3839e58-18a9-4f1d-a85a-e06f1f3348ae None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:41.469327 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3839e58-18a9-4f1d-a85a-e06f1f3348ae None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:41.469327 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3839e58-18a9-4f1d-a85a-e06f1f3348ae None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:41.469948 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3839e58-18a9-4f1d-a85a-e06f1f3348ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:41.470247 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3839e58-18a9-4f1d-a85a-e06f1f3348ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:41.471619 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3839e58-18a9-4f1d-a85a-e06f1f3348ae None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:41.479400 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 619/1264] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:10:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:41.485826 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a178fcc-5a98-4da2-9a94-1d6dc627fa7a None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:41.552008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:41.584596 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a178fcc-5a98-4da2-9a94-1d6dc627fa7a None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:41.618313 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f0a23a-69e5-4625-afd2-7719cbfc0a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:41.618947 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f0a23a-69e5-4625-afd2-7719cbfc0a5a None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:41.619226 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f0a23a-69e5-4625-afd2-7719cbfc0a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:41.619496 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f0a23a-69e5-4625-afd2-7719cbfc0a5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:41.630973 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f0a23a-69e5-4625-afd2-7719cbfc0a5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:41.632396 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f0a23a-69e5-4625-afd2-7719cbfc0a5a None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:41.646521 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-e2f0a23a-69e5-4625-afd2-7719cbfc0a5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:10:41.646521 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:10:41.646521 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:10:41.646521 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:10:41.646521 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:10:41.646521 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:10:41.646521 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:10:41.646521 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:10:41.646521 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:10:41.646521 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:10:41.646521 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:10:41.646521 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:10:41.646521 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:10:41.646521 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:10:41.646521 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:10:41.647487 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:10:41.647487 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:10:41.647487 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:10:41.647487 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:10:41.647487 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:10:41.647487 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:10:41.647487 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:10:41.647487 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:10:41.647487 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:10:41.647487 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:10:41.647487 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:10:41.647487 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:10:41.647487 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:10:41.647487 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:10:41.647487 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:10:41.647487 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:10:41.647487 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:10:41.648630 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:10:41.648630 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:10:41.648630 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:10:41.648630 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:10:41.648630 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 646/1265] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:10:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:10:41.654277 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3839e58-18a9-4f1d-a85a-e06f1f3348ae None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:41.737559 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:41.757724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3839e58-18a9-4f1d-a85a-e06f1f3348ae None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:41.791094 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18f9bd-4330-43c2-9787-b9e14d4c98fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:41.791713 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18f9bd-4330-43c2-9787-b9e14d4c98fe None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:41.791968 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18f9bd-4330-43c2-9787-b9e14d4c98fe None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:41.792235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18f9bd-4330-43c2-9787-b9e14d4c98fe None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:41.793015 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18f9bd-4330-43c2-9787-b9e14d4c98fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:41.793318 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18f9bd-4330-43c2-9787-b9e14d4c98fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:41.794674 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18f9bd-4330-43c2-9787-b9e14d4c98fe None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:41.805379 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 620/1266] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:10:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:41.812304 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f0a23a-69e5-4625-afd2-7719cbfc0a5a None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:41.910858 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f0a23a-69e5-4625-afd2-7719cbfc0a5a None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:41.930711 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:41.945471 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082fe98c-5ba6-424c-875b-4a88b601d471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:41.946097 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082fe98c-5ba6-424c-875b-4a88b601d471 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:41.946371 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082fe98c-5ba6-424c-875b-4a88b601d471 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:41.946636 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082fe98c-5ba6-424c-875b-4a88b601d471 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:41.954247 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082fe98c-5ba6-424c-875b-4a88b601d471 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:41.955724 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082fe98c-5ba6-424c-875b-4a88b601d471 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:41.966770 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-082fe98c-5ba6-424c-875b-4a88b601d471 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 05 14:10:41.966770 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:10:41.966770 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:10:41.966770 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:10:41.966770 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:10:41.966770 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:10:41.966770 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:10:41.966770 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:10:41.966770 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:10:41.966770 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:10:41.966770 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:10:41.966770 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:10:41.966770 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 14:10:41.966770 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 14:10:41.966770 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 14:10:41.966770 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 14:10:41.967907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:10:41.967907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:10:41.967907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:10:41.967907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:10:41.967907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:10:41.967907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:10:41.967907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:10:41.967907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:10:41.967907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:10:41.967907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:10:41.967907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:10:41.967907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:10:41.967907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:10:41.967907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:10:41.967907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:10:41.967907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:10:41.967907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:10:41.967907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:10:41.968903 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:10:41.968903 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:10:41.968903 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 14:10:41.968903 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 14:10:41.968903 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 14:10:41.968903 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 14:10:41.968903 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 14:10:41.968903 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 14:10:41.968903 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 05 14:10:41.968903 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:10:41.968903 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 647/1267] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:10:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:10:41.974032 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18f9bd-4330-43c2-9787-b9e14d4c98fe None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:42.072801 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18f9bd-4330-43c2-9787-b9e14d4c98fe None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:42.106842 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91035f2-b6d4-4ba9-8c21-24a4c235a382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:42.107600 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91035f2-b6d4-4ba9-8c21-24a4c235a382 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:42.107886 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91035f2-b6d4-4ba9-8c21-24a4c235a382 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:42.108158 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91035f2-b6d4-4ba9-8c21-24a4c235a382 None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:42.108985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91035f2-b6d4-4ba9-8c21-24a4c235a382 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:42.109296 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91035f2-b6d4-4ba9-8c21-24a4c235a382 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:42.110712 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91035f2-b6d4-4ba9-8c21-24a4c235a382 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:42.119874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:42.128268 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 621/1268] 10.222.0.26 () {56 vars in 1143 bytes} [Fri Aug 5 14:10:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:42.134146 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082fe98c-5ba6-424c-875b-4a88b601d471 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:42.233182 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082fe98c-5ba6-424c-875b-4a88b601d471 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:42.265998 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dadb03-8f13-4c00-9e1a-3a52df5b584a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:42.266623 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dadb03-8f13-4c00-9e1a-3a52df5b584a None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:42.266901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dadb03-8f13-4c00-9e1a-3a52df5b584a None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:42.267173 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dadb03-8f13-4c00-9e1a-3a52df5b584a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:42.280607 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dadb03-8f13-4c00-9e1a-3a52df5b584a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:42.282041 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dadb03-8f13-4c00-9e1a-3a52df5b584a None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:42.296611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-27dadb03-8f13-4c00-9e1a-3a52df5b584a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:10:42.296611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:10:42.296611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:10:42.296611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:10:42.296611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:10:42.296611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:10:42.296611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:10:42.296611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:10:42.296611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:10:42.296611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:10:42.296611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:10:42.296611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:10:42.296611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:10:42.296611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:10:42.296611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:10:42.297662 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:10:42.297662 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:10:42.297662 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:10:42.297662 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:10:42.297662 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:10:42.297662 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:10:42.297662 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:10:42.297662 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:10:42.297662 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:10:42.297662 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:10:42.297662 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:10:42.297662 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:10:42.297662 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:10:42.297662 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:10:42.297662 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:10:42.297662 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:10:42.297662 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:10:42.298918 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:10:42.298918 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:10:42.298918 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:10:42.298918 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:10:42.298918 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 648/1269] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:10:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:10:42.303512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91035f2-b6d4-4ba9-8c21-24a4c235a382 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:42.402223 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91035f2-b6d4-4ba9-8c21-24a4c235a382 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:42.436141 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a89cef-ef9c-404b-8d1c-e2533b260fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:42.436775 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a89cef-ef9c-404b-8d1c-e2533b260fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:42.437046 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a89cef-ef9c-404b-8d1c-e2533b260fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:42.437309 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a89cef-ef9c-404b-8d1c-e2533b260fe0 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:42.437991 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a89cef-ef9c-404b-8d1c-e2533b260fe0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:42.438289 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a89cef-ef9c-404b-8d1c-e2533b260fe0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:42.439629 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a89cef-ef9c-404b-8d1c-e2533b260fe0 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:42.451320 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 622/1270] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:10:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:42.457139 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dadb03-8f13-4c00-9e1a-3a52df5b584a None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:42.488628 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:42.555919 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dadb03-8f13-4c00-9e1a-3a52df5b584a None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:42.591170 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f83a2a-017f-4f75-9714-f45c5211bcf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:42.591811 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f83a2a-017f-4f75-9714-f45c5211bcf7 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:42.592093 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f83a2a-017f-4f75-9714-f45c5211bcf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:42.592361 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f83a2a-017f-4f75-9714-f45c5211bcf7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:42.598836 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f83a2a-017f-4f75-9714-f45c5211bcf7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:42.600264 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f83a2a-017f-4f75-9714-f45c5211bcf7 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:42.612461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-c5f83a2a-017f-4f75-9714-f45c5211bcf7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:10:42.612461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:10:42.612461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:10:42.612461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:10:42.612461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:10:42.612461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:10:42.612461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:10:42.612461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:10:42.612461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:10:42.612461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:10:42.612461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:10:42.612461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:10:42.612461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 14:10:42.612461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 14:10:42.612461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 14:10:42.613693 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 14:10:42.613693 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:10:42.613693 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:10:42.613693 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:10:42.613693 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:10:42.613693 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:10:42.613693 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:10:42.613693 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:10:42.613693 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:10:42.613693 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:10:42.613693 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:10:42.613693 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:10:42.613693 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:10:42.613693 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:10:42.613693 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:10:42.613693 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:10:42.613693 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:10:42.614832 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 14:10:42.614832 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 14:10:42.614832 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:10:42.614832 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:10:42.614832 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:10:42.614832 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:10:42.614832 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:10:42.614832 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:10:42.614832 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 649/1271] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:10:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:10:42.619671 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a89cef-ef9c-404b-8d1c-e2533b260fe0 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:42.676942 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:42.716329 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a89cef-ef9c-404b-8d1c-e2533b260fe0 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:42.748692 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b134c-e434-4541-853c-cce6feec7190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:42.749246 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b134c-e434-4541-853c-cce6feec7190 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:42.749519 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b134c-e434-4541-853c-cce6feec7190 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:42.749787 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b134c-e434-4541-853c-cce6feec7190 None admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:42.750521 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b134c-e434-4541-853c-cce6feec7190 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:42.750823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b134c-e434-4541-853c-cce6feec7190 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:42.752236 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b134c-e434-4541-853c-cce6feec7190 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:42.764310 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 623/1272] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Aug 5 14:10:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:42.770424 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f83a2a-017f-4f75-9714-f45c5211bcf7 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:42.862125 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:42.870968 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f83a2a-017f-4f75-9714-f45c5211bcf7 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:42.902875 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99f539c-4cac-40b2-9547-8ef77a6c418e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:42.903527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f539c-4cac-40b2-9547-8ef77a6c418e None admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:42.903800 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f539c-4cac-40b2-9547-8ef77a6c418e None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:42.904066 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f539c-4cac-40b2-9547-8ef77a6c418e None admin] PATH_INFO: `/v3/projects/319973e011c54b4ca1f2198c8a5e135b/users/fff761f6f81a46098f94bd2ec2be9c1a/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:42.905985 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99f539c-4cac-40b2-9547-8ef77a6c418e None admin] RBAC: Authorizing `identity:create_grant(project_id=319973e011c54b4ca1f2198c8a5e135b, user_id=fff761f6f81a46098f94bd2ec2be9c1a, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:42.907531 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99f539c-4cac-40b2-9547-8ef77a6c418e None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:43.023431 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 650/1273] 10.222.0.26 () {58 vars in 1439 bytes} [Fri Aug 5 14:10:42 2022] PUT /identity/v3/projects/319973e011c54b4ca1f2198c8a5e135b/users/fff761f6f81a46098f94bd2ec2be9c1a/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:10:45.053252 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1887d0a-e7c0-4038-aeb4-4cf5121199b8 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:45.053803 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1887d0a-e7c0-4038-aeb4-4cf5121199b8 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:45.053803 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1887d0a-e7c0-4038-aeb4-4cf5121199b8 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:45.055129 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 624/1274] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:10:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:10:45.060684 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376b558-8c03-40f1-b4f3-9eee4bbc3a78 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:45.061086 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376b558-8c03-40f1-b4f3-9eee4bbc3a78 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:45.061267 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376b558-8c03-40f1-b4f3-9eee4bbc3a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:45.079547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-9376b558-8c03-40f1-b4f3-9eee4bbc3a78 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:10:45.080762 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-9376b558-8c03-40f1-b4f3-9eee4bbc3a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:10:45.146794 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 651/1275] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:10:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:10:45.170505 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1887d0a-e7c0-4038-aeb4-4cf5121199b8 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:45.271057 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1887d0a-e7c0-4038-aeb4-4cf5121199b8 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:45.289148 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:45.305258 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f8a535-2f6d-4bb0-8c7f-3b93b5b24328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:45.305831 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8a535-2f6d-4bb0-8c7f-3b93b5b24328 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:45.306107 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8a535-2f6d-4bb0-8c7f-3b93b5b24328 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:45.306378 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f8a535-2f6d-4bb0-8c7f-3b93b5b24328 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:45.316642 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f8a535-2f6d-4bb0-8c7f-3b93b5b24328 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 14:10:45.321507 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-22f8a535-2f6d-4bb0-8c7f-3b93b5b24328 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:10:45.321507 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:10:45.321507 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:10:45.321507 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:10:45.321507 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:10:45.321507 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:10:45.321507 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:10:45.321507 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:10:45.321507 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:10:45.321507 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:10:45.321507 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:10:45.321507 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:10:45.321507 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 14:10:45.321507 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 14:10:45.321507 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 14:10:45.321507 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 14:10:45.322633 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 14:10:45.322633 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 14:10:45.322633 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 14:10:45.322633 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 14:10:45.322633 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:10:45.322633 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:10:45.322633 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:10:45.322633 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:10:45.322633 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:10:45.322633 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:10:45.322633 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:10:45.322633 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:10:45.322633 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:10:45.322633 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:10:45.322633 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:10:45.322633 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:10:45.324235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:10:45.324235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:10:45.324235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:10:45.324235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:10:45.324235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 14:10:45.324235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 14:10:45.324235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 14:10:45.324235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 14:10:45.324235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 14:10:45.324235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 14:10:45.324235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 14:10:45.324235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:10:45.324235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 625/1276] 10.222.0.26 () {56 vars in 1096 bytes} [Fri Aug 5 14:10:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:10:45.329597 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9376b558-8c03-40f1-b4f3-9eee4bbc3a78 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:45.409798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:45.427496 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9376b558-8c03-40f1-b4f3-9eee4bbc3a78 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:45.462490 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1c012-b712-4e48-9b1f-e4209565a66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:45.463094 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1c012-b712-4e48-9b1f-e4209565a66f None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:45.463369 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1c012-b712-4e48-9b1f-e4209565a66f None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:45.463765 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1c012-b712-4e48-9b1f-e4209565a66f None admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:45.464647 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa1c012-b712-4e48-9b1f-e4209565a66f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:45.468928 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa1c012-b712-4e48-9b1f-e4209565a66f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:45.468928 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa1c012-b712-4e48-9b1f-e4209565a66f None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:45.475195 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 652/1277] 10.222.0.26 () {56 vars in 1099 bytes} [Fri Aug 5 14:10:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:45.481146 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f8a535-2f6d-4bb0-8c7f-3b93b5b24328 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:45.580976 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f8a535-2f6d-4bb0-8c7f-3b93b5b24328 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:45.614364 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc85c11a-6687-4e28-908c-130ab7bb9ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:45.614996 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc85c11a-6687-4e28-908c-130ab7bb9ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:45.615271 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc85c11a-6687-4e28-908c-130ab7bb9ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:45.615567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc85c11a-6687-4e28-908c-130ab7bb9ed2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:45.625862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc85c11a-6687-4e28-908c-130ab7bb9ed2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:45.627298 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc85c11a-6687-4e28-908c-130ab7bb9ed2 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:45.642711 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-bc85c11a-6687-4e28-908c-130ab7bb9ed2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:10:45.642711 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:10:45.642711 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:10:45.642711 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:10:45.642711 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:10:45.642711 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:10:45.642711 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:10:45.642711 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:10:45.642711 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:10:45.642711 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:10:45.642711 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:10:45.642711 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:10:45.642711 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:10:45.642711 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:10:45.642711 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:10:45.643823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:10:45.643823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:10:45.643823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:10:45.643823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:10:45.643823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:10:45.643823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:10:45.643823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:10:45.643823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:10:45.643823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:10:45.643823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:10:45.643823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:10:45.643823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:10:45.643823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:10:45.643823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:10:45.643823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:10:45.643823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:10:45.643823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:10:45.644975 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:10:45.644975 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:10:45.644975 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:10:45.644975 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:10:45.644975 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 626/1278] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:10:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:10:45.649824 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1c012-b712-4e48-9b1f-e4209565a66f None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:45.749723 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1c012-b712-4e48-9b1f-e4209565a66f None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:45.782191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac69fa9-43cb-4be1-afc6-07869cbe5119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:45.782796 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac69fa9-43cb-4be1-afc6-07869cbe5119 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:45.783118 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac69fa9-43cb-4be1-afc6-07869cbe5119 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:45.783390 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac69fa9-43cb-4be1-afc6-07869cbe5119 None admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:45.784068 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac69fa9-43cb-4be1-afc6-07869cbe5119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:45.784445 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac69fa9-43cb-4be1-afc6-07869cbe5119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:45.785858 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac69fa9-43cb-4be1-afc6-07869cbe5119 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:45.797668 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 653/1279] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:10:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:45.803452 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc85c11a-6687-4e28-908c-130ab7bb9ed2 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:45.834621 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:45.903245 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc85c11a-6687-4e28-908c-130ab7bb9ed2 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:45.935383 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767a13a-1a50-4823-9b02-b7f8a4cafc34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:45.936052 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767a13a-1a50-4823-9b02-b7f8a4cafc34 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:45.936327 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767a13a-1a50-4823-9b02-b7f8a4cafc34 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:45.936618 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767a13a-1a50-4823-9b02-b7f8a4cafc34 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:45.943395 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2767a13a-1a50-4823-9b02-b7f8a4cafc34 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:45.944849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2767a13a-1a50-4823-9b02-b7f8a4cafc34 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:45.956871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-2767a13a-1a50-4823-9b02-b7f8a4cafc34 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 05 14:10:45.956871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:10:45.956871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:10:45.956871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:10:45.956871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:10:45.956871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:10:45.956871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:10:45.956871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:10:45.956871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:10:45.956871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:10:45.956871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:10:45.956871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:10:45.956871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 14:10:45.956871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 14:10:45.956871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 14:10:45.956871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 14:10:45.957830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:10:45.957830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:10:45.957830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 14:10:45.957830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:10:45.957830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 14:10:45.957830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 14:10:45.957830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:10:45.957830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:10:45.957830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:10:45.957830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:10:45.957830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:10:45.957830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:10:45.957830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:10:45.957830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:10:45.957830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:10:45.957830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:10:45.957830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:10:45.957830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:10:45.958877 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:10:45.958877 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:10:45.958877 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 14:10:45.958877 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 14:10:45.958877 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 14:10:45.958877 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 14:10:45.958877 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 14:10:45.958877 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 14:10:45.958877 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 05 14:10:45.958877 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:10:45.958877 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 627/1280] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:10:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:10:45.964909 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac69fa9-43cb-4be1-afc6-07869cbe5119 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:45.968828 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:46.062712 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac69fa9-43cb-4be1-afc6-07869cbe5119 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:46.096153 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa2632-e379-4b5d-96a6-df3a7b226675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:46.096729 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa2632-e379-4b5d-96a6-df3a7b226675 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:46.097009 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa2632-e379-4b5d-96a6-df3a7b226675 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:46.097280 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa2632-e379-4b5d-96a6-df3a7b226675 None admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:46.098091 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa2632-e379-4b5d-96a6-df3a7b226675 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:46.098364 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa2632-e379-4b5d-96a6-df3a7b226675 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:46.099766 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa2632-e379-4b5d-96a6-df3a7b226675 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:46.117256 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 654/1281] 10.222.0.26 () {56 vars in 1143 bytes} [Fri Aug 5 14:10:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:46.123820 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767a13a-1a50-4823-9b02-b7f8a4cafc34 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:46.215376 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:46.224477 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767a13a-1a50-4823-9b02-b7f8a4cafc34 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:46.257906 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba80c59-56d4-4b35-8809-05fac0f04cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:46.258515 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba80c59-56d4-4b35-8809-05fac0f04cc9 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:46.258789 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba80c59-56d4-4b35-8809-05fac0f04cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:46.259055 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba80c59-56d4-4b35-8809-05fac0f04cc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:46.333816 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba80c59-56d4-4b35-8809-05fac0f04cc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:46.335354 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba80c59-56d4-4b35-8809-05fac0f04cc9 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:46.426773 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:46.436984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 628/1282] 10.222.0.26 () {58 vars in 1282 bytes} [Fri Aug 5 14:10:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 14:10:46.442936 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa2632-e379-4b5d-96a6-df3a7b226675 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:46.506502 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:46.547761 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa2632-e379-4b5d-96a6-df3a7b226675 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:46.580806 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902687f-9141-4ebf-ab22-c733387844b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:46.581435 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902687f-9141-4ebf-ab22-c733387844b9 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:46.581714 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902687f-9141-4ebf-ab22-c733387844b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:46.581984 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902687f-9141-4ebf-ab22-c733387844b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:46.591654 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902687f-9141-4ebf-ab22-c733387844b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:46.593125 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902687f-9141-4ebf-ab22-c733387844b9 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:46.607725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-0902687f-9141-4ebf-ab22-c733387844b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:10:46.607725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:10:46.607725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:10:46.607725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:10:46.607725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:10:46.607725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:10:46.607725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:10:46.607725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:10:46.607725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:10:46.607725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:10:46.607725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:10:46.607725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:10:46.607725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 14:10:46.607725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 14:10:46.607725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 14:10:46.608803 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 14:10:46.608803 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:10:46.608803 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:10:46.608803 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:10:46.608803 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:10:46.608803 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:10:46.608803 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:10:46.608803 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:10:46.608803 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:10:46.608803 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:10:46.608803 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:10:46.608803 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:10:46.608803 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:10:46.608803 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:10:46.608803 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:10:46.608803 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:10:46.608803 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:10:46.609913 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 14:10:46.609913 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 14:10:46.609913 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 14:10:46.609913 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:10:46.609913 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 655/1283] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:10:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 14:10:46.615007 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba80c59-56d4-4b35-8809-05fac0f04cc9 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:46.712966 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba80c59-56d4-4b35-8809-05fac0f04cc9 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:46.746643 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a219551b-42ad-4797-9b8a-7f7c09c48b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:46.747274 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219551b-42ad-4797-9b8a-7f7c09c48b65 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:46.747549 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219551b-42ad-4797-9b8a-7f7c09c48b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:46.747806 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219551b-42ad-4797-9b8a-7f7c09c48b65 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:46.748509 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a219551b-42ad-4797-9b8a-7f7c09c48b65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:46.748810 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a219551b-42ad-4797-9b8a-7f7c09c48b65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:46.750268 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a219551b-42ad-4797-9b8a-7f7c09c48b65 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:46.761229 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 629/1284] 10.222.0.26 () {56 vars in 1109 bytes} [Fri Aug 5 14:10:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:46.767275 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902687f-9141-4ebf-ab22-c733387844b9 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:46.866891 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902687f-9141-4ebf-ab22-c733387844b9 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:46.880453 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:46.899588 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab74164-f63b-4edd-9467-2e0c28c5f3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:46.900223 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab74164-f63b-4edd-9467-2e0c28c5f3bb None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:46.900546 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab74164-f63b-4edd-9467-2e0c28c5f3bb None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:46.900819 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab74164-f63b-4edd-9467-2e0c28c5f3bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:46.907883 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab74164-f63b-4edd-9467-2e0c28c5f3bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:46.909334 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab74164-f63b-4edd-9467-2e0c28c5f3bb None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:46.921147 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application [None req-bab74164-f63b-4edd-9467-2e0c28c5f3bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:10:46.921147 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:10:46.921147 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:10:46.921147 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:10:46.921147 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:10:46.921147 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:10:46.921147 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:10:46.921147 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:10:46.921147 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:10:46.921147 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:10:46.921147 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:10:46.921147 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:10:46.921147 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 14:10:46.921147 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 14:10:46.921147 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 14:10:46.922208 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 14:10:46.922208 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:10:46.922208 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:10:46.922208 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:10:46.922208 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:10:46.922208 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:10:46.922208 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:10:46.922208 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:10:46.922208 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application with Lock( Aug 05 14:10:46.922208 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:10:46.922208 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:10:46.922208 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:10:46.922208 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:10:46.922208 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:10:46.922208 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:10:46.922208 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:10:46.922208 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:10:46.923473 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 14:10:46.923473 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 14:10:46.923473 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 14:10:46.923473 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 14:10:46.923473 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 14:10:46.923473 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 14:10:46.923473 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 14:10:46.923473 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: ERROR keystone.server.flask.application Aug 05 14:10:46.923473 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 656/1285] 10.222.0.26 () {56 vars in 1111 bytes} [Fri Aug 5 14:10:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:10:46.928176 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a219551b-42ad-4797-9b8a-7f7c09c48b65 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:46.977882 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:47.027598 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a219551b-42ad-4797-9b8a-7f7c09c48b65 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:47.061102 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4500822-ab6b-4dd6-a323-41bb2a5e17de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:47.061708 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4500822-ab6b-4dd6-a323-41bb2a5e17de None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:47.061984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4500822-ab6b-4dd6-a323-41bb2a5e17de None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:47.062252 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4500822-ab6b-4dd6-a323-41bb2a5e17de None admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:47.062995 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4500822-ab6b-4dd6-a323-41bb2a5e17de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:47.063296 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4500822-ab6b-4dd6-a323-41bb2a5e17de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:47.064786 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4500822-ab6b-4dd6-a323-41bb2a5e17de None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:47.076597 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 630/1286] 10.222.0.26 () {56 vars in 1148 bytes} [Fri Aug 5 14:10:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:47.082867 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab74164-f63b-4edd-9467-2e0c28c5f3bb None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:47.181413 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab74164-f63b-4edd-9467-2e0c28c5f3bb None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:47.215047 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5ecc5-abe9-4098-af3d-bd99168a6590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:47.215653 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5ecc5-abe9-4098-af3d-bd99168a6590 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:47.215929 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5ecc5-abe9-4098-af3d-bd99168a6590 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:47.216200 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5ecc5-abe9-4098-af3d-bd99168a6590 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:47.217023 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e5ecc5-abe9-4098-af3d-bd99168a6590 None admin] RBAC: Adding query filter params (role.id=299c794c7acb4a5ca36288c3191aad10, scope.project.id=319973e011c54b4ca1f2198c8a5e135b, user.id=fff761f6f81a46098f94bd2ec2be9c1a) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:47.217331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e5ecc5-abe9-4098-af3d-bd99168a6590 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:47.218741 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e5ecc5-abe9-4098-af3d-bd99168a6590 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:47.224373 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 657/1287] 10.222.0.26 () {56 vars in 1374 bytes} [Fri Aug 5 14:10:47 2022] GET /identity/v3/role_assignments?user.id=fff761f6f81a46098f94bd2ec2be9c1a&scope.project.id=319973e011c54b4ca1f2198c8a5e135b&role.id=299c794c7acb4a5ca36288c3191aad10 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:49.250556 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5e30f-1117-4dae-8c64-fbedd3346a65 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:49.251138 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5e30f-1117-4dae-8c64-fbedd3346a65 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:49.251138 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5e30f-1117-4dae-8c64-fbedd3346a65 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:49.252653 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 631/1288] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:10:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:10:49.258759 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63566dfb-52e2-4288-a0f7-08870b9921bf None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:49.259285 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63566dfb-52e2-4288-a0f7-08870b9921bf None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:49.259285 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63566dfb-52e2-4288-a0f7-08870b9921bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:49.279830 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-63566dfb-52e2-4288-a0f7-08870b9921bf None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:10:49.281094 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-63566dfb-52e2-4288-a0f7-08870b9921bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:10:49.314864 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:49.353617 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 658/1289] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:10:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:10:49.377323 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a5e30f-1117-4dae-8c64-fbedd3346a65 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:49.485806 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a5e30f-1117-4dae-8c64-fbedd3346a65 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:49.498740 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:49.522211 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdfb1b5-8507-4faa-ba3e-6d052ab1febf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:49.522783 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfb1b5-8507-4faa-ba3e-6d052ab1febf None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:49.523063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfb1b5-8507-4faa-ba3e-6d052ab1febf None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:49.523336 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfb1b5-8507-4faa-ba3e-6d052ab1febf None admin] PATH_INFO: `/v3/services/placement` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:49.523969 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdfb1b5-8507-4faa-ba3e-6d052ab1febf None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:49.525378 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdfb1b5-8507-4faa-ba3e-6d052ab1febf None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:49.534851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-6bdfb1b5-8507-4faa-ba3e-6d052ab1febf None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 14:10:49.534851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:10:49.534851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 14:10:49.534851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 14:10:49.534851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 14:10:49.534851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 14:10:49.534851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 14:10:49.534851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:10:49.534851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 14:10:49.534851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:10:49.534851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 14:10:49.534851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:10:49.534851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:10:49.534851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:10:49.534851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:10:49.534851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:10:49.534851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:10:49.537826 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:10:49.537826 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:10:49.537826 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:10:49.537826 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:10:49.537826 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:10:49.537826 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:10:49.537826 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 14:10:49.537826 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 14:10:49.537826 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 14:10:49.537826 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 14:10:49.537826 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:10:49.537826 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:10:49.537826 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:10:49.537826 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:10:49.537826 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:10:49.537826 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:10:49.538682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:10:49.538682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:10:49.538682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:10:49.538682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:10:49.538682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:10:49.538682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:10:49.538682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:10:49.538682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:10:49.538682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:10:49.538682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:10:49.538682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 14:10:49.538682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:10:49.538682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 14:10:49.538682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:10:49.538682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 632/1290] 10.222.0.26 () {56 vars in 1117 bytes} [Fri Aug 5 14:10:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:10:49.544041 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63566dfb-52e2-4288-a0f7-08870b9921bf None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:49.646798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63566dfb-52e2-4288-a0f7-08870b9921bf None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:49.683214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f685d580-98f8-4bb8-abea-9ef01f147fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:49.683835 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f685d580-98f8-4bb8-abea-9ef01f147fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:49.684212 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f685d580-98f8-4bb8-abea-9ef01f147fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:49.684517 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f685d580-98f8-4bb8-abea-9ef01f147fc4 None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:49.685230 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f685d580-98f8-4bb8-abea-9ef01f147fc4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:49.685536 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f685d580-98f8-4bb8-abea-9ef01f147fc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:49.686891 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f685d580-98f8-4bb8-abea-9ef01f147fc4 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:49.693925 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 659/1291] 10.222.0.26 () {56 vars in 1116 bytes} [Fri Aug 5 14:10:49 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:49.699631 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdfb1b5-8507-4faa-ba3e-6d052ab1febf None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:49.799559 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdfb1b5-8507-4faa-ba3e-6d052ab1febf None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:49.833201 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6371e7c7-534a-462a-a0ae-90be229f3bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:49.833807 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371e7c7-534a-462a-a0ae-90be229f3bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:49.834081 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371e7c7-534a-462a-a0ae-90be229f3bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:49.834349 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371e7c7-534a-462a-a0ae-90be229f3bf2 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:49.835011 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6371e7c7-534a-462a-a0ae-90be229f3bf2 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:49.835315 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6371e7c7-534a-462a-a0ae-90be229f3bf2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:49.836697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6371e7c7-534a-462a-a0ae-90be229f3bf2 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:49.841578 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 633/1292] 10.222.0.26 () {56 vars in 1116 bytes} [Fri Aug 5 14:10:49 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:51.836567 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f39dc4-abcf-4e81-9dee-b424fbee4865 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:51.837191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f39dc4-abcf-4e81-9dee-b424fbee4865 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:51.837191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f39dc4-abcf-4e81-9dee-b424fbee4865 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:51.838990 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 660/1293] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:10:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:10:51.844260 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a62ec2-8270-4a94-9af1-9a72f9abb1e5 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:51.844713 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a62ec2-8270-4a94-9af1-9a72f9abb1e5 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:51.844828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a62ec2-8270-4a94-9af1-9a72f9abb1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:51.863146 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-03a62ec2-8270-4a94-9af1-9a72f9abb1e5 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:10:51.864336 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-03a62ec2-8270-4a94-9af1-9a72f9abb1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:10:51.893653 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:51.930337 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 634/1294] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:10:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:10:51.955269 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f39dc4-abcf-4e81-9dee-b424fbee4865 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:51.985954 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:52.054941 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f39dc4-abcf-4e81-9dee-b424fbee4865 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:52.088702 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fefa6b6-9900-4bf3-ac71-9316679f88b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:52.089309 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fefa6b6-9900-4bf3-ac71-9316679f88b7 None admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:52.089583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fefa6b6-9900-4bf3-ac71-9316679f88b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:52.089856 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fefa6b6-9900-4bf3-ac71-9316679f88b7 None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:52.090540 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fefa6b6-9900-4bf3-ac71-9316679f88b7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:52.091887 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fefa6b6-9900-4bf3-ac71-9316679f88b7 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:52.176380 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 661/1295] 10.222.0.26 () {60 vars in 1141 bytes} [Fri Aug 5 14:10:51 2022] POST /identity/v3/services => generated 246 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:10:54.176174 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2923445-7f74-4436-b56c-d6ab8d2e5db6 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:54.176784 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2923445-7f74-4436-b56c-d6ab8d2e5db6 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:54.176784 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2923445-7f74-4436-b56c-d6ab8d2e5db6 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:54.178118 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 635/1296] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:10:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:10:54.184060 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e14066-8342-41ae-830f-ac1e1902dc0f None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:54.184498 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e14066-8342-41ae-830f-ac1e1902dc0f None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:54.184645 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e14066-8342-41ae-830f-ac1e1902dc0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:54.202365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-58e14066-8342-41ae-830f-ac1e1902dc0f None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:10:54.203538 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-58e14066-8342-41ae-830f-ac1e1902dc0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:10:54.273932 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 662/1297] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:10:54 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:10:54.297182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2923445-7f74-4436-b56c-d6ab8d2e5db6 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:54.394801 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2923445-7f74-4436-b56c-d6ab8d2e5db6 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:54.427811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c07fd5e-e656-4cb0-837b-aaaf6371d1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:54.428376 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07fd5e-e656-4cb0-837b-aaaf6371d1cd None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:54.428678 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07fd5e-e656-4cb0-837b-aaaf6371d1cd None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:54.428943 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07fd5e-e656-4cb0-837b-aaaf6371d1cd None admin] PATH_INFO: `/v3/services/placement` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:54.429594 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c07fd5e-e656-4cb0-837b-aaaf6371d1cd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:54.430932 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c07fd5e-e656-4cb0-837b-aaaf6371d1cd None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:54.439828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-3c07fd5e-e656-4cb0-837b-aaaf6371d1cd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 14:10:54.439828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:10:54.439828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 14:10:54.439828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 14:10:54.439828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 14:10:54.439828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 14:10:54.439828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 14:10:54.439828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:10:54.439828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 14:10:54.439828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:10:54.439828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 14:10:54.439828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:10:54.439828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:10:54.439828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:10:54.439828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:10:54.439828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:10:54.439828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:10:54.444814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:10:54.444814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:10:54.444814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:10:54.444814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:10:54.444814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:10:54.444814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:10:54.444814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 14:10:54.444814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 14:10:54.444814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 14:10:54.444814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 14:10:54.444814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:10:54.444814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:10:54.444814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:10:54.444814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:10:54.444814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:10:54.444814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:10:54.445964 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:10:54.445964 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:10:54.445964 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:10:54.445964 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:10:54.445964 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:10:54.445964 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:10:54.445964 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:10:54.445964 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:10:54.445964 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:10:54.445964 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:10:54.445964 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 14:10:54.445964 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:10:54.445964 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 14:10:54.445964 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:10:54.445964 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 636/1298] 10.222.0.26 () {56 vars in 1117 bytes} [Fri Aug 5 14:10:54 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:10:54.448175 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e14066-8342-41ae-830f-ac1e1902dc0f None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:54.466412 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:54.544680 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e14066-8342-41ae-830f-ac1e1902dc0f None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:54.576995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2584f-d882-44e3-8dd1-7b5b5ebd2af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:54.577633 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2584f-d882-44e3-8dd1-7b5b5ebd2af9 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:54.577911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2584f-d882-44e3-8dd1-7b5b5ebd2af9 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:54.578179 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2584f-d882-44e3-8dd1-7b5b5ebd2af9 None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:54.578880 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c2584f-d882-44e3-8dd1-7b5b5ebd2af9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:54.579178 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c2584f-d882-44e3-8dd1-7b5b5ebd2af9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:54.580563 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c2584f-d882-44e3-8dd1-7b5b5ebd2af9 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:54.585444 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 663/1299] 10.222.0.26 () {56 vars in 1116 bytes} [Fri Aug 5 14:10:54 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:54.592492 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c07fd5e-e656-4cb0-837b-aaaf6371d1cd None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:54.624046 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:54.690267 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c07fd5e-e656-4cb0-837b-aaaf6371d1cd None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:54.724393 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678f294-9745-4e2e-a660-8464cf57dc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:54.725056 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678f294-9745-4e2e-a660-8464cf57dc61 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:54.725330 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678f294-9745-4e2e-a660-8464cf57dc61 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:54.725596 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678f294-9745-4e2e-a660-8464cf57dc61 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:54.726337 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8678f294-9745-4e2e-a660-8464cf57dc61 None admin] RBAC: Adding query filter params (interface=public, service_id=e763cdc65e814b22ae7708dbcd0cf77b, region_id=RegionOne) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:54.726698 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8678f294-9745-4e2e-a660-8464cf57dc61 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:54.728024 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8678f294-9745-4e2e-a660-8464cf57dc61 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:54.733908 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 637/1300] 10.222.0.26 () {56 vars in 1251 bytes} [Fri Aug 5 14:10:54 2022] GET /identity/v3/endpoints?service_id=e763cdc65e814b22ae7708dbcd0cf77b&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:54.739941 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2584f-d882-44e3-8dd1-7b5b5ebd2af9 None admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:54.820956 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:54.840556 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2584f-d882-44e3-8dd1-7b5b5ebd2af9 None admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:54.874139 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951125e-7cff-4a4e-b5c3-52c17275d53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:54.874762 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951125e-7cff-4a4e-b5c3-52c17275d53d None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:54.875414 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951125e-7cff-4a4e-b5c3-52c17275d53d None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:54.875687 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951125e-7cff-4a4e-b5c3-52c17275d53d None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:54.876337 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f951125e-7cff-4a4e-b5c3-52c17275d53d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:54.877715 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f951125e-7cff-4a4e-b5c3-52c17275d53d None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:54.884665 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 664/1301] 10.222.0.26 () {56 vars in 1087 bytes} [Fri Aug 5 14:10:54 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:10:56.894147 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901e469-b10c-43a8-afbb-c4db4e530cce None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:56.894674 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901e469-b10c-43a8-afbb-c4db4e530cce None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:56.894674 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901e469-b10c-43a8-afbb-c4db4e530cce None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:56.896020 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 638/1302] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:10:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:10:56.901754 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d8dc3-177c-43b8-8da3-c2ad1d5abbe7 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:56.902205 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d8dc3-177c-43b8-8da3-c2ad1d5abbe7 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:56.902300 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d8dc3-177c-43b8-8da3-c2ad1d5abbe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:56.921110 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-194d8dc3-177c-43b8-8da3-c2ad1d5abbe7 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:10:56.922294 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-194d8dc3-177c-43b8-8da3-c2ad1d5abbe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:10:56.987224 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 665/1303] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:10:56 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 14:10:57.010438 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0901e469-b10c-43a8-afbb-c4db4e530cce None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:57.103150 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:57.110414 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0901e469-b10c-43a8-afbb-c4db4e530cce None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:57.143465 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efebda3-bfdc-4340-bb20-9d73a616d658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:57.144088 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efebda3-bfdc-4340-bb20-9d73a616d658 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:57.144434 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efebda3-bfdc-4340-bb20-9d73a616d658 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:57.144654 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efebda3-bfdc-4340-bb20-9d73a616d658 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:57.145268 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efebda3-bfdc-4340-bb20-9d73a616d658 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:57.146644 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efebda3-bfdc-4340-bb20-9d73a616d658 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:57.155859 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-2efebda3-bfdc-4340-bb20-9d73a616d658 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 14:10:57.155859 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:10:57.155859 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 14:10:57.155859 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 14:10:57.155859 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 14:10:57.155859 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 14:10:57.155859 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 14:10:57.155859 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:10:57.155859 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 14:10:57.155859 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:10:57.155859 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 14:10:57.155859 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:10:57.155859 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 14:10:57.155859 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 14:10:57.155859 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 14:10:57.155859 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 14:10:57.155859 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 14:10:57.157042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 14:10:57.157042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 14:10:57.157042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 14:10:57.157042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 14:10:57.157042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 14:10:57.157042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 14:10:57.157042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 14:10:57.157042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 14:10:57.157042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 14:10:57.157042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 14:10:57.157042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 14:10:57.157042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 14:10:57.157042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 14:10:57.157042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 14:10:57.157042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 14:10:57.157042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 14:10:57.157904 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 14:10:57.157904 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Aug 05 14:10:57.157904 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 14:10:57.157904 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Aug 05 14:10:57.157904 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 14:10:57.157904 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 14:10:57.157904 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 14:10:57.157904 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Aug 05 14:10:57.157904 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 14:10:57.157904 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Aug 05 14:10:57.157904 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 14:10:57.157904 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 14:10:57.157904 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 14:10:57.157904 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Aug 05 14:10:57.157904 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 639/1304] 10.222.0.26 () {56 vars in 1117 bytes} [Fri Aug 5 14:10:57 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 14:10:57.163035 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d8dc3-177c-43b8-8da3-c2ad1d5abbe7 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:57.237893 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:10:57.260563 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d8dc3-177c-43b8-8da3-c2ad1d5abbe7 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:57.295034 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cf3e4c-e82b-44ee-acfc-879c15e49e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:57.295646 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf3e4c-e82b-44ee-acfc-879c15e49e64 None admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:57.295924 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf3e4c-e82b-44ee-acfc-879c15e49e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:57.296193 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf3e4c-e82b-44ee-acfc-879c15e49e64 None admin] PATH_INFO: `/v3/services` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:57.296894 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cf3e4c-e82b-44ee-acfc-879c15e49e64 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:10:57.297194 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cf3e4c-e82b-44ee-acfc-879c15e49e64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:57.298585 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cf3e4c-e82b-44ee-acfc-879c15e49e64 None admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:57.303615 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 666/1305] 10.222.0.26 () {56 vars in 1116 bytes} [Fri Aug 5 14:10:57 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:10:57.310120 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efebda3-bfdc-4340-bb20-9d73a616d658 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:10:57.408194 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efebda3-bfdc-4340-bb20-9d73a616d658 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:10:57.441406 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec6fe01-23c1-4ca1-8418-c125da22ae6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:10:57.442012 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6fe01-23c1-4ca1-8418-c125da22ae6c None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:10:57.442287 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6fe01-23c1-4ca1-8418-c125da22ae6c None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:10:57.442556 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6fe01-23c1-4ca1-8418-c125da22ae6c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:10:57.443261 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec6fe01-23c1-4ca1-8418-c125da22ae6c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:10:57.444652 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec6fe01-23c1-4ca1-8418-c125da22ae6c None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:10:57.533674 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 640/1306] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:10:57 2022] POST /identity/v3/endpoints => generated 331 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:11:07.300573 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=30 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:11:07.529009 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:17:07.788665 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b5c45-d68b-42c8-8f44-d24e74be0501 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:07.789139 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b5c45-d68b-42c8-8f44-d24e74be0501 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:07.789244 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b5c45-d68b-42c8-8f44-d24e74be0501 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:07.790532 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 667/1307] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:17:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:17:07.798937 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7db39-7c61-4667-a332-2c62a5e164ac None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:07.800027 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7db39-7c61-4667-a332-2c62a5e164ac None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:07.800875 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7db39-7c61-4667-a332-2c62a5e164ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:07.898741 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-a3d7db39-7c61-4667-a332-2c62a5e164ac None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:17:07.900257 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a3d7db39-7c61-4667-a332-2c62a5e164ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:08.032868 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 641/1308] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:17:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:17:08.084567 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9cddf-b4c2-4743-96a2-de18f5367093 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:08.084926 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9cddf-b4c2-4743-96a2-de18f5367093 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:08.085099 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9cddf-b4c2-4743-96a2-de18f5367093 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:08.087464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 668/1309] 10.222.0.26 () {54 vars in 965 bytes} [Fri Aug 5 14:17:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:17:08.096594 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d841865-5666-42f2-8ccd-6eafa2a496d0 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:08.096886 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d841865-5666-42f2-8ccd-6eafa2a496d0 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:08.097198 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d841865-5666-42f2-8ccd-6eafa2a496d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:08.201383 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-5d841865-5666-42f2-8ccd-6eafa2a496d0 None None] MFA Rules not processed for user `695e37560f8f495e987d1bd17acc2680`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:17:08.203495 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5d841865-5666-42f2-8ccd-6eafa2a496d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:08.228030 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:17:08.284522 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 642/1310] 10.222.0.26 () {58 vars in 1062 bytes} [Fri Aug 5 14:17:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:17:08.293372 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9cddf-b4c2-4743-96a2-de18f5367093 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:17:08.295224 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-57b9cddf-b4c2-4743-96a2-de18f5367093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:08.416083 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9cddf-b4c2-4743-96a2-de18f5367093 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:17:08.492522 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe965632-58d5-413f-92f7-343cac97bb2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:17:08.493153 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe965632-58d5-413f-92f7-343cac97bb2a service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:08.493403 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe965632-58d5-413f-92f7-343cac97bb2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:08.493670 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe965632-58d5-413f-92f7-343cac97bb2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:08.495274 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-fe965632-58d5-413f-92f7-343cac97bb2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:08.559746 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:17:08.615394 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe965632-58d5-413f-92f7-343cac97bb2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:17:08.616898 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe965632-58d5-413f-92f7-343cac97bb2a service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:17:08.635947 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 669/1311] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:17:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:17:10.912808 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33998c27-52d9-48f0-82e9-e16131553f66 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:10.913351 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33998c27-52d9-48f0-82e9-e16131553f66 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:10.913351 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33998c27-52d9-48f0-82e9-e16131553f66 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:10.914733 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 643/1312] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:17:10.919840 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f83af-8c07-4454-963d-db92a320ba96 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:10.920228 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f83af-8c07-4454-963d-db92a320ba96 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:10.920386 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f83af-8c07-4454-963d-db92a320ba96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:11.007231 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-991f83af-8c07-4454-963d-db92a320ba96 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:17:11.008479 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-991f83af-8c07-4454-963d-db92a320ba96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:11.114543 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 670/1313] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:17:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:17:11.152530 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33998c27-52d9-48f0-82e9-e16131553f66 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:17:11.154084 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-33998c27-52d9-48f0-82e9-e16131553f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:11.257285 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33998c27-52d9-48f0-82e9-e16131553f66 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:17:11.452221 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:17:11.455352 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499badbb-f36c-456a-b5c5-e0213faf1431 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:17:11.455964 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499badbb-f36c-456a-b5c5-e0213faf1431 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:11.456249 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499badbb-f36c-456a-b5c5-e0213faf1431 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:11.456533 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499badbb-f36c-456a-b5c5-e0213faf1431 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:11.458258 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-499badbb-f36c-456a-b5c5-e0213faf1431 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:11.564843 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499badbb-f36c-456a-b5c5-e0213faf1431 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:17:11.566268 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499badbb-f36c-456a-b5c5-e0213faf1431 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:17:11.583771 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 644/1314] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:17:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:17:14.260636 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11598d63-b436-455d-b6ce-49a45540f3df None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:14.261180 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11598d63-b436-455d-b6ce-49a45540f3df None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:14.261180 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11598d63-b436-455d-b6ce-49a45540f3df None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:14.262530 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 671/1315] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:17:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:17:14.267847 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56483d54-87e8-49b8-8e14-29ce880471c3 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:14.268259 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56483d54-87e8-49b8-8e14-29ce880471c3 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:14.268382 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56483d54-87e8-49b8-8e14-29ce880471c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:14.333567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-56483d54-87e8-49b8-8e14-29ce880471c3 None None] MFA Rules not processed for user `da4b0c24aa284fd59788ab74c424af63`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:17:14.334753 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-56483d54-87e8-49b8-8e14-29ce880471c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:14.386107 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:17:14.429981 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 645/1316] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:17:14 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:17:14.484592 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81314af-f3d3-47c1-90ba-ead0e9059070 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:14.484874 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81314af-f3d3-47c1-90ba-ead0e9059070 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:14.485153 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81314af-f3d3-47c1-90ba-ead0e9059070 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:14.486766 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 672/1317] 10.222.0.26 () {54 vars in 965 bytes} [Fri Aug 5 14:17:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:17:14.492700 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79dcef-d96c-46a8-8a91-02b159451ad7 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:14.492986 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79dcef-d96c-46a8-8a91-02b159451ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:14.493258 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79dcef-d96c-46a8-8a91-02b159451ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:14.568932 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-ca79dcef-d96c-46a8-8a91-02b159451ad7 None None] MFA Rules not processed for user `695e37560f8f495e987d1bd17acc2680`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:17:14.570133 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ca79dcef-d96c-46a8-8a91-02b159451ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:14.649638 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 646/1318] 10.222.0.26 () {58 vars in 1062 bytes} [Fri Aug 5 14:17:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:17:14.659425 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81314af-f3d3-47c1-90ba-ead0e9059070 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:17:14.660998 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-b81314af-f3d3-47c1-90ba-ead0e9059070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:14.670579 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:17:14.764699 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81314af-f3d3-47c1-90ba-ead0e9059070 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:17:14.836565 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa0983f-664e-44e4-b768-f74f802b5281 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:17:14.837172 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa0983f-664e-44e4-b768-f74f802b5281 service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:14.837445 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa0983f-664e-44e4-b768-f74f802b5281 service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:14.837714 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa0983f-664e-44e4-b768-f74f802b5281 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:14.839320 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-bfa0983f-664e-44e4-b768-f74f802b5281 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:14.899875 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:17:14.944693 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa0983f-664e-44e4-b768-f74f802b5281 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:17:14.946104 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa0983f-664e-44e4-b768-f74f802b5281 service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:17:14.964354 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 673/1319] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:17:14 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:17:18.105828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d290c9c-1829-4c2b-84a2-6157ca9f07b6 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:18.106427 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d290c9c-1829-4c2b-84a2-6157ca9f07b6 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:18.106427 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d290c9c-1829-4c2b-84a2-6157ca9f07b6 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:18.107708 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 647/1320] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:17:18.113005 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b18ccc-892d-44fb-97a8-12246d7c582d None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:18.113471 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b18ccc-892d-44fb-97a8-12246d7c582d None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:18.113574 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b18ccc-892d-44fb-97a8-12246d7c582d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:18.179219 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-17b18ccc-892d-44fb-97a8-12246d7c582d None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:17:18.180435 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-17b18ccc-892d-44fb-97a8-12246d7c582d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:18.275492 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 674/1321] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:17:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:17:18.314061 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d290c9c-1829-4c2b-84a2-6157ca9f07b6 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:17:18.315616 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-0d290c9c-1829-4c2b-84a2-6157ca9f07b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:18.364985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:17:18.416474 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d290c9c-1829-4c2b-84a2-6157ca9f07b6 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:17:18.486540 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea445b5f-320b-4563-9b0f-219f04beee86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:17:18.487141 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea445b5f-320b-4563-9b0f-219f04beee86 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:18.487409 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea445b5f-320b-4563-9b0f-219f04beee86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:18.487672 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea445b5f-320b-4563-9b0f-219f04beee86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:18.489286 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ea445b5f-320b-4563-9b0f-219f04beee86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:18.581689 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:17:18.595046 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea445b5f-320b-4563-9b0f-219f04beee86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:17:18.596502 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea445b5f-320b-4563-9b0f-219f04beee86 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:17:18.614771 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 648/1322] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:17:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:17:21.710694 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca6a2f-e518-4107-95d1-ad9c03ba97d6 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:21.711202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca6a2f-e518-4107-95d1-ad9c03ba97d6 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:21.711202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca6a2f-e518-4107-95d1-ad9c03ba97d6 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:21.712518 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 675/1323] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:17:21.717607 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e19814-86ad-4183-b4ef-db6b278d49db None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:21.718559 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e19814-86ad-4183-b4ef-db6b278d49db None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:21.718700 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e19814-86ad-4183-b4ef-db6b278d49db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:21.783366 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-38e19814-86ad-4183-b4ef-db6b278d49db None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:17:21.784595 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-38e19814-86ad-4183-b4ef-db6b278d49db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:21.881798 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 649/1324] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:17:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:17:21.918303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca6a2f-e518-4107-95d1-ad9c03ba97d6 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:17:21.919922 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-cbca6a2f-e518-4107-95d1-ad9c03ba97d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:21.944071 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:17:22.037461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca6a2f-e518-4107-95d1-ad9c03ba97d6 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:17:22.118190 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac32b037-9d0c-4fcc-9a2b-ba2220617e81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:17:22.118807 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac32b037-9d0c-4fcc-9a2b-ba2220617e81 service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:22.119086 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac32b037-9d0c-4fcc-9a2b-ba2220617e81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:22.119362 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac32b037-9d0c-4fcc-9a2b-ba2220617e81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:22.121004 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-ac32b037-9d0c-4fcc-9a2b-ba2220617e81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:22.192040 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:17:22.230553 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac32b037-9d0c-4fcc-9a2b-ba2220617e81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:17:22.231980 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac32b037-9d0c-4fcc-9a2b-ba2220617e81 service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:17:22.250294 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 676/1325] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:17:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:17:24.333981 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba44cd4-dbde-4f12-b670-c8cc8dda0afa None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:24.334532 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba44cd4-dbde-4f12-b670-c8cc8dda0afa None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:24.334532 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba44cd4-dbde-4f12-b670-c8cc8dda0afa None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:24.335907 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 650/1326] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:17:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:17:24.341266 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ef246-db18-439e-b00b-b353d37dba54 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:24.341682 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ef246-db18-439e-b00b-b353d37dba54 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:24.341810 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ef246-db18-439e-b00b-b353d37dba54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:24.409062 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-0c2ef246-db18-439e-b00b-b353d37dba54 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:17:24.410176 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-0c2ef246-db18-439e-b00b-b353d37dba54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:24.505586 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 677/1327] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:17:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:17:24.541560 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba44cd4-dbde-4f12-b670-c8cc8dda0afa None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:17:24.543090 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-2ba44cd4-dbde-4f12-b670-c8cc8dda0afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:24.603648 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:17:24.651113 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba44cd4-dbde-4f12-b670-c8cc8dda0afa None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:17:24.729116 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb49ef-5ce4-4611-8c31-bdd836109d21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:17:24.729732 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb49ef-5ce4-4611-8c31-bdd836109d21 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:24.730004 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb49ef-5ce4-4611-8c31-bdd836109d21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:24.730267 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb49ef-5ce4-4611-8c31-bdd836109d21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:24.731868 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d5cb49ef-5ce4-4611-8c31-bdd836109d21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:24.841894 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:17:24.844968 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cb49ef-5ce4-4611-8c31-bdd836109d21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:17:24.846413 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cb49ef-5ce4-4611-8c31-bdd836109d21 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:17:24.865356 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 651/1328] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:17:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:17:26.930202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434c98b-1294-4bfb-be1c-ba870d9cb7d1 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:26.930788 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434c98b-1294-4bfb-be1c-ba870d9cb7d1 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:26.930788 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434c98b-1294-4bfb-be1c-ba870d9cb7d1 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:26.932136 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 678/1329] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:17:26.937743 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b22bfd-70ec-4cf6-85f4-8fab27b9c00a None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:26.938152 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b22bfd-70ec-4cf6-85f4-8fab27b9c00a None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:26.938304 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b22bfd-70ec-4cf6-85f4-8fab27b9c00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:27.003537 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-94b22bfd-70ec-4cf6-85f4-8fab27b9c00a None None] MFA Rules not processed for user `da4b0c24aa284fd59788ab74c424af63`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:17:27.004776 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-94b22bfd-70ec-4cf6-85f4-8fab27b9c00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:27.104245 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 652/1330] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:17:26 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:17:27.142997 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434c98b-1294-4bfb-be1c-ba870d9cb7d1 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:17:27.144699 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-e434c98b-1294-4bfb-be1c-ba870d9cb7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:27.177604 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:17:27.257979 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434c98b-1294-4bfb-be1c-ba870d9cb7d1 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:17:27.330944 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b6d560-33ce-403c-9aae-a0f97aaad86c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:17:27.331570 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6d560-33ce-403c-9aae-a0f97aaad86c service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:27.331841 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6d560-33ce-403c-9aae-a0f97aaad86c service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:27.332110 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6d560-33ce-403c-9aae-a0f97aaad86c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:27.333813 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-f9b6d560-33ce-403c-9aae-a0f97aaad86c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:27.413346 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:17:27.445451 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b6d560-33ce-403c-9aae-a0f97aaad86c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:17:27.446904 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b6d560-33ce-403c-9aae-a0f97aaad86c service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:17:27.465897 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 679/1331] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:17:27 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 14:17:30.042220 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290c073-d030-4c7a-ab0f-c0225e43fc00 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:30.042780 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290c073-d030-4c7a-ab0f-c0225e43fc00 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:30.042780 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290c073-d030-4c7a-ab0f-c0225e43fc00 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:30.044001 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 653/1332] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:17:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:17:30.049887 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75696cb-7eb5-4525-a10b-794d671018b5 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:30.050338 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75696cb-7eb5-4525-a10b-794d671018b5 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:30.050445 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75696cb-7eb5-4525-a10b-794d671018b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:30.115879 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-e75696cb-7eb5-4525-a10b-794d671018b5 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:17:30.117106 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-e75696cb-7eb5-4525-a10b-794d671018b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:30.211805 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 680/1333] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:17:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:17:30.249004 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290c073-d030-4c7a-ab0f-c0225e43fc00 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:17:30.250593 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-1290c073-d030-4c7a-ab0f-c0225e43fc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:30.319766 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:17:30.353279 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290c073-d030-4c7a-ab0f-c0225e43fc00 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:17:30.424014 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b44f65-16cd-47fd-b8f5-31a3c89a8d40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:17:30.424655 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b44f65-16cd-47fd-b8f5-31a3c89a8d40 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:30.424933 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b44f65-16cd-47fd-b8f5-31a3c89a8d40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:30.425197 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b44f65-16cd-47fd-b8f5-31a3c89a8d40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:30.430436 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-04b44f65-16cd-47fd-b8f5-31a3c89a8d40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:30.533378 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b44f65-16cd-47fd-b8f5-31a3c89a8d40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:17:30.534802 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b44f65-16cd-47fd-b8f5-31a3c89a8d40 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:17:30.544534 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:17:30.552863 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 654/1334] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:17:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:17:32.591948 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02ef21-08ca-4942-9c8f-b0323b409e68 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:32.592518 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02ef21-08ca-4942-9c8f-b0323b409e68 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:32.593058 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02ef21-08ca-4942-9c8f-b0323b409e68 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:32.593967 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 681/1335] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:17:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:17:32.599279 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4d370-b1f2-48f9-9981-f8edf1009c4b None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:32.599706 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4d370-b1f2-48f9-9981-f8edf1009c4b None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:32.599802 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4d370-b1f2-48f9-9981-f8edf1009c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:32.664271 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-40d4d370-b1f2-48f9-9981-f8edf1009c4b None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:17:32.665484 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-40d4d370-b1f2-48f9-9981-f8edf1009c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:32.766451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 655/1336] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:17:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:17:32.807071 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf02ef21-08ca-4942-9c8f-b0323b409e68 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:17:32.808630 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-cf02ef21-08ca-4942-9c8f-b0323b409e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:32.853378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:17:32.916621 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf02ef21-08ca-4942-9c8f-b0323b409e68 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:17:32.989221 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9319d13-3f97-43b3-8419-26dd40fc921e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:17:32.992348 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9319d13-3f97-43b3-8419-26dd40fc921e service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:32.992348 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9319d13-3f97-43b3-8419-26dd40fc921e service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:32.992348 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9319d13-3f97-43b3-8419-26dd40fc921e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:32.992348 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-d9319d13-3f97-43b3-8419-26dd40fc921e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:33.074580 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:17:33.100089 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9319d13-3f97-43b3-8419-26dd40fc921e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:17:33.101566 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9319d13-3f97-43b3-8419-26dd40fc921e service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:17:33.120264 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 682/1337] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:17:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:17:35.690824 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650555d9-7a0f-4c80-b848-0f6f00c4303e None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:35.691345 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650555d9-7a0f-4c80-b848-0f6f00c4303e None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:35.691345 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650555d9-7a0f-4c80-b848-0f6f00c4303e None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:35.692673 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 656/1338] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:17:35.697846 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d4257-9560-417d-ad75-06496b6000f8 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:35.698273 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d4257-9560-417d-ad75-06496b6000f8 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:35.698392 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d4257-9560-417d-ad75-06496b6000f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:35.763701 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-a39d4257-9560-417d-ad75-06496b6000f8 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:17:35.765713 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-a39d4257-9560-417d-ad75-06496b6000f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:35.860238 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 683/1339] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:17:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:17:35.896680 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650555d9-7a0f-4c80-b848-0f6f00c4303e None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:17:35.898208 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-650555d9-7a0f-4c80-b848-0f6f00c4303e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:35.973447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:17:36.001460 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650555d9-7a0f-4c80-b848-0f6f00c4303e None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:17:36.074512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1cde5-b44a-48c4-bc86-77dbdf539cd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:17:36.075119 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1cde5-b44a-48c4-bc86-77dbdf539cd8 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:36.075395 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1cde5-b44a-48c4-bc86-77dbdf539cd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:36.075664 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1cde5-b44a-48c4-bc86-77dbdf539cd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:36.077389 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-72d1cde5-b44a-48c4-bc86-77dbdf539cd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:36.185802 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1cde5-b44a-48c4-bc86-77dbdf539cd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:17:36.187718 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1cde5-b44a-48c4-bc86-77dbdf539cd8 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:17:36.206004 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 657/1340] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:17:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:17:40.455749 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdedde0-d267-4927-b29d-1f03d509e00f None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:40.456253 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdedde0-d267-4927-b29d-1f03d509e00f None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:40.456348 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdedde0-d267-4927-b29d-1f03d509e00f None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:40.457684 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 684/1341] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:17:40.462870 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cb7e9-0d42-4655-b56e-ca0d55b61b52 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:40.463314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cb7e9-0d42-4655-b56e-ca0d55b61b52 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:40.463536 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cb7e9-0d42-4655-b56e-ca0d55b61b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:40.469766 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:17:40.533174 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-c56cb7e9-0d42-4655-b56e-ca0d55b61b52 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:17:40.534412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c56cb7e9-0d42-4655-b56e-ca0d55b61b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:40.630552 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 658/1342] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:17:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:17:40.669687 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdedde0-d267-4927-b29d-1f03d509e00f None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:17:40.671251 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-1fdedde0-d267-4927-b29d-1f03d509e00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:40.719505 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:17:40.774628 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdedde0-d267-4927-b29d-1f03d509e00f None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:17:40.848080 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83bb2d-c0cf-4e0a-89d4-ecbc96cd4514 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:17:40.848731 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83bb2d-c0cf-4e0a-89d4-ecbc96cd4514 service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:40.849009 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83bb2d-c0cf-4e0a-89d4-ecbc96cd4514 service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:40.849282 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83bb2d-c0cf-4e0a-89d4-ecbc96cd4514 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:40.850893 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-ca83bb2d-c0cf-4e0a-89d4-ecbc96cd4514 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:40.944648 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:17:40.960856 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca83bb2d-c0cf-4e0a-89d4-ecbc96cd4514 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:17:40.962279 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca83bb2d-c0cf-4e0a-89d4-ecbc96cd4514 service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:17:40.980480 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 685/1343] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:17:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:17:43.933992 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945fbd4d-3f6c-4a49-b29a-8fc4ad075f1c None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:43.934547 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945fbd4d-3f6c-4a49-b29a-8fc4ad075f1c None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:43.934752 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945fbd4d-3f6c-4a49-b29a-8fc4ad075f1c None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:43.936109 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 659/1344] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:17:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:17:43.941970 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4a3fc-c6dc-4723-a3d9-bd7e73964144 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:43.942378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4a3fc-c6dc-4723-a3d9-bd7e73964144 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:43.942530 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4a3fc-c6dc-4723-a3d9-bd7e73964144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:44.009638 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-2ff4a3fc-c6dc-4723-a3d9-bd7e73964144 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:17:44.010851 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-2ff4a3fc-c6dc-4723-a3d9-bd7e73964144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:44.112039 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 686/1345] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:17:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:17:44.149185 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945fbd4d-3f6c-4a49-b29a-8fc4ad075f1c None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:17:44.150723 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-945fbd4d-3f6c-4a49-b29a-8fc4ad075f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:44.238532 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:17:44.255387 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945fbd4d-3f6c-4a49-b29a-8fc4ad075f1c None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:17:44.329913 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4adffc-e693-4183-8cbc-87f8b3aedc4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:17:44.330560 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4adffc-e693-4183-8cbc-87f8b3aedc4a service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:44.330833 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4adffc-e693-4183-8cbc-87f8b3aedc4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:44.331097 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4adffc-e693-4183-8cbc-87f8b3aedc4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:44.332770 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-fd4adffc-e693-4183-8cbc-87f8b3aedc4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:44.443453 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4adffc-e693-4183-8cbc-87f8b3aedc4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:17:44.444921 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4adffc-e693-4183-8cbc-87f8b3aedc4a service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:17:44.463661 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 660/1346] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:17:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:17:48.651075 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb7d32d-72d0-4c47-862e-675fb267a7ed None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:48.651599 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb7d32d-72d0-4c47-862e-675fb267a7ed None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:48.651599 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb7d32d-72d0-4c47-862e-675fb267a7ed None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:48.654390 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 687/1347] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:17:48.659473 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920252ba-3460-47a0-a5ee-9c93d9be32b1 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:48.659473 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920252ba-3460-47a0-a5ee-9c93d9be32b1 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:48.659973 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920252ba-3460-47a0-a5ee-9c93d9be32b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:48.687838 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:17:48.742747 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-920252ba-3460-47a0-a5ee-9c93d9be32b1 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:17:48.744100 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-920252ba-3460-47a0-a5ee-9c93d9be32b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:48.848099 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 661/1348] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:17:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:17:48.887310 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb7d32d-72d0-4c47-862e-675fb267a7ed None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:17:48.888888 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-8eb7d32d-72d0-4c47-862e-675fb267a7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:48.949749 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:17:48.998177 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb7d32d-72d0-4c47-862e-675fb267a7ed None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:17:49.078994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3731d345-44f3-4863-9cc0-f55bdfbc7941 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:17:49.079610 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731d345-44f3-4863-9cc0-f55bdfbc7941 service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:49.079886 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731d345-44f3-4863-9cc0-f55bdfbc7941 service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:49.080153 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731d345-44f3-4863-9cc0-f55bdfbc7941 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:49.081801 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-3731d345-44f3-4863-9cc0-f55bdfbc7941 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:49.195387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:17:49.198717 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3731d345-44f3-4863-9cc0-f55bdfbc7941 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:17:49.200193 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3731d345-44f3-4863-9cc0-f55bdfbc7941 service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:17:49.219019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 688/1349] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:17:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:17:54.126591 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdd0c4-2890-44e8-9986-d7a987ee4762 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:54.127618 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdd0c4-2890-44e8-9986-d7a987ee4762 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:54.128208 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdd0c4-2890-44e8-9986-d7a987ee4762 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:54.130353 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 662/1350] 10.222.0.26 () {54 vars in 921 bytes} [Fri Aug 5 14:17:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:17:54.135044 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc36ad4-71dd-4f4c-a12a-c06ff02bf14a None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:54.135704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc36ad4-71dd-4f4c-a12a-c06ff02bf14a None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:54.135704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc36ad4-71dd-4f4c-a12a-c06ff02bf14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:54.282409 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-6bc36ad4-71dd-4f4c-a12a-c06ff02bf14a None None] MFA Rules not processed for user `fff761f6f81a46098f94bd2ec2be9c1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:17:54.283615 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-6bc36ad4-71dd-4f4c-a12a-c06ff02bf14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:54.411673 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 689/1351] 10.222.0.26 () {58 vars in 1018 bytes} [Fri Aug 5 14:17:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:17:58.548674 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73527a-bb34-4de0-bb10-bc8c634cc7ca None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:58.549218 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73527a-bb34-4de0-bb10-bc8c634cc7ca None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:58.549354 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73527a-bb34-4de0-bb10-bc8c634cc7ca None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:58.550922 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 663/1352] 10.222.0.26 () {54 vars in 921 bytes} [Fri Aug 5 14:17:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:17:58.556261 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb82d23-1d19-4875-9c33-d7d7628c2e5d None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:58.556743 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb82d23-1d19-4875-9c33-d7d7628c2e5d None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:58.556988 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb82d23-1d19-4875-9c33-d7d7628c2e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:58.613056 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:17:58.636256 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-bcb82d23-1d19-4875-9c33-d7d7628c2e5d None None] MFA Rules not processed for user `fff761f6f81a46098f94bd2ec2be9c1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:17:58.637500 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-bcb82d23-1d19-4875-9c33-d7d7628c2e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:17:58.751881 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 690/1353] 10.222.0.26 () {58 vars in 1018 bytes} [Fri Aug 5 14:17:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:17:58.846167 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=59 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:17:59.994520 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b589e-b92a-4856-8bd7-b095c639f34b None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:17:59.995136 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b589e-b92a-4856-8bd7-b095c639f34b None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:17:59.995136 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b589e-b92a-4856-8bd7-b095c639f34b None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:17:59.996429 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 664/1354] 10.222.0.26 () {54 vars in 921 bytes} [Fri Aug 5 14:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:18:00.007215 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8fc54b-21fe-4f67-877c-a351904088c2 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:00.007215 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8fc54b-21fe-4f67-877c-a351904088c2 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:00.007215 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8fc54b-21fe-4f67-877c-a351904088c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:00.082955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-1b8fc54b-21fe-4f67-877c-a351904088c2 None None] MFA Rules not processed for user `fff761f6f81a46098f94bd2ec2be9c1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:18:00.084176 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-1b8fc54b-21fe-4f67-877c-a351904088c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:00.128251 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:00.195646 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 691/1355] 10.222.0.26 () {58 vars in 1018 bytes} [Fri Aug 5 14:18:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:18:03.665540 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd479f1-ae0b-4643-a25d-2e62cc0db1b4 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:03.666106 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd479f1-ae0b-4643-a25d-2e62cc0db1b4 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:03.666106 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd479f1-ae0b-4643-a25d-2e62cc0db1b4 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:03.667615 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 665/1356] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:18:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:18:03.673125 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e700b8f6-d05c-42d4-bccd-4bbde47af727 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:03.673976 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e700b8f6-d05c-42d4-bccd-4bbde47af727 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:03.674255 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e700b8f6-d05c-42d4-bccd-4bbde47af727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:03.756459 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-e700b8f6-d05c-42d4-bccd-4bbde47af727 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:18:03.758120 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-e700b8f6-d05c-42d4-bccd-4bbde47af727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:03.862257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 692/1357] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:18:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:18:03.917008 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a1dc5-e9da-48ba-9f0d-8c6b9db551c2 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:03.917232 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a1dc5-e9da-48ba-9f0d-8c6b9db551c2 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:03.917738 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a1dc5-e9da-48ba-9f0d-8c6b9db551c2 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:03.919017 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 666/1358] 10.222.0.26 () {54 vars in 962 bytes} [Fri Aug 5 14:18:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:18:03.928713 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1423b81-f1dc-44de-bbed-6f818c89cb65 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:03.928999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1423b81-f1dc-44de-bbed-6f818c89cb65 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:03.929271 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1423b81-f1dc-44de-bbed-6f818c89cb65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:03.946675 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:04.009494 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-c1423b81-f1dc-44de-bbed-6f818c89cb65 None None] MFA Rules not processed for user `5ec4940c99fc4fa8a5bc7cdabc4558fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:18:04.010735 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-c1423b81-f1dc-44de-bbed-6f818c89cb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:04.131597 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 693/1359] 10.222.0.26 () {58 vars in 1059 bytes} [Fri Aug 5 14:18:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:18:04.139859 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375a1dc5-e9da-48ba-9f0d-8c6b9db551c2 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:18:04.141390 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-375a1dc5-e9da-48ba-9f0d-8c6b9db551c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:04.292333 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375a1dc5-e9da-48ba-9f0d-8c6b9db551c2 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:18:04.377772 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:04.417808 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdd5ef1-6760-4910-aafb-fe8863fd4ffb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:18:04.418429 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd5ef1-6760-4910-aafb-fe8863fd4ffb service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:04.418700 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd5ef1-6760-4910-aafb-fe8863fd4ffb service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:04.418974 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd5ef1-6760-4910-aafb-fe8863fd4ffb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:04.421200 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-3cdd5ef1-6760-4910-aafb-fe8863fd4ffb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:04.515773 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdd5ef1-6760-4910-aafb-fe8863fd4ffb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:18:04.517517 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdd5ef1-6760-4910-aafb-fe8863fd4ffb service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:18:04.537504 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 667/1360] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:18:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:18:06.769307 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9926d6-2e5e-4025-978d-080ffe69af39 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:06.769809 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9926d6-2e5e-4025-978d-080ffe69af39 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:06.769809 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9926d6-2e5e-4025-978d-080ffe69af39 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:06.771212 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 694/1361] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:18:06.776155 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476af6a-52c9-44e6-9f39-2ee1f3625e4b None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:06.776613 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476af6a-52c9-44e6-9f39-2ee1f3625e4b None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:06.776795 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476af6a-52c9-44e6-9f39-2ee1f3625e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:06.852986 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-4476af6a-52c9-44e6-9f39-2ee1f3625e4b None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:18:06.854695 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-4476af6a-52c9-44e6-9f39-2ee1f3625e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:06.897308 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:06.964461 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 668/1362] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:18:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:18:07.009621 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9926d6-2e5e-4025-978d-080ffe69af39 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:18:07.011186 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-8a9926d6-2e5e-4025-978d-080ffe69af39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:07.092013 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:07.184215 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9926d6-2e5e-4025-978d-080ffe69af39 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:18:07.338145 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfcbef8-91e0-4cea-a5a0-6387bb86b6fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:18:07.338833 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcbef8-91e0-4cea-a5a0-6387bb86b6fc service nova] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:07.339177 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcbef8-91e0-4cea-a5a0-6387bb86b6fc service nova] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:07.339506 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcbef8-91e0-4cea-a5a0-6387bb86b6fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:07.341199 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-6dfcbef8-91e0-4cea-a5a0-6387bb86b6fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:07.368301 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:07.452284 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfcbef8-91e0-4cea-a5a0-6387bb86b6fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:18:07.454011 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfcbef8-91e0-4cea-a5a0-6387bb86b6fc service nova] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:18:07.475752 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 695/1363] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:18:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:18:09.978548 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd42082-3d26-4112-9a6b-0723b4bb8662 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:09.979737 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd42082-3d26-4112-9a6b-0723b4bb8662 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:09.980182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd42082-3d26-4112-9a6b-0723b4bb8662 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:09.982096 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 669/1364] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:18:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:18:09.988470 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa034a1a-2770-4ef9-bf1a-6c411dd221ec None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:09.988952 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa034a1a-2770-4ef9-bf1a-6c411dd221ec None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:09.989268 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa034a1a-2770-4ef9-bf1a-6c411dd221ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:10.063315 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-fa034a1a-2770-4ef9-bf1a-6c411dd221ec None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:18:10.064640 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-fa034a1a-2770-4ef9-bf1a-6c411dd221ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:10.174767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:10.185102 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 696/1365] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:18:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:18:10.226013 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbb536-603d-4ecd-b2ba-b062db5a922b None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:10.226452 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbb536-603d-4ecd-b2ba-b062db5a922b None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:10.226794 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbb536-603d-4ecd-b2ba-b062db5a922b None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:10.228571 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 670/1366] 10.222.0.26 () {54 vars in 962 bytes} [Fri Aug 5 14:18:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:18:10.234756 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cf61b-7ccb-4975-9c0b-ad89094db048 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:10.235109 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cf61b-7ccb-4975-9c0b-ad89094db048 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:10.235453 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cf61b-7ccb-4975-9c0b-ad89094db048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:10.319847 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-903cf61b-7ccb-4975-9c0b-ad89094db048 None None] MFA Rules not processed for user `5ec4940c99fc4fa8a5bc7cdabc4558fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:18:10.321185 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-903cf61b-7ccb-4975-9c0b-ad89094db048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:10.459903 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 697/1367] 10.222.0.26 () {58 vars in 1059 bytes} [Fri Aug 5 14:18:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:18:10.468064 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bbb536-603d-4ecd-b2ba-b062db5a922b None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:18:10.469879 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-06bbb536-603d-4ecd-b2ba-b062db5a922b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:10.633026 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bbb536-603d-4ecd-b2ba-b062db5a922b None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:18:10.655048 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:10.753601 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40f9fa8-30ce-4b80-957b-cc7ac6e7325c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:18:10.754133 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f9fa8-30ce-4b80-957b-cc7ac6e7325c service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:10.754401 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f9fa8-30ce-4b80-957b-cc7ac6e7325c service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:10.754675 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f9fa8-30ce-4b80-957b-cc7ac6e7325c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:10.756364 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-b40f9fa8-30ce-4b80-957b-cc7ac6e7325c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:10.850798 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40f9fa8-30ce-4b80-957b-cc7ac6e7325c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:18:10.852392 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40f9fa8-30ce-4b80-957b-cc7ac6e7325c service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:18:10.874797 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 671/1368] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:18:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:18:11.823196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc899931-1bef-4388-a612-057426b9fb9a None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:11.823728 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc899931-1bef-4388-a612-057426b9fb9a None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:11.823728 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc899931-1bef-4388-a612-057426b9fb9a None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:11.825194 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 698/1369] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:18:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:18:11.830949 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186dc07b-ee48-4fde-a959-b73da616a89e None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:11.831376 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186dc07b-ee48-4fde-a959-b73da616a89e None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:11.831500 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186dc07b-ee48-4fde-a959-b73da616a89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:11.864828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:11.900229 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-186dc07b-ee48-4fde-a959-b73da616a89e None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:18:11.901489 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-186dc07b-ee48-4fde-a959-b73da616a89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:12.005512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 672/1370] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:18:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:18:12.347497 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b3039-0a1c-4828-9ed2-3320480a3170 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:12.347497 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b3039-0a1c-4828-9ed2-3320480a3170 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:12.347845 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b3039-0a1c-4828-9ed2-3320480a3170 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:12.349077 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 699/1371] 10.222.0.26 () {54 vars in 964 bytes} [Fri Aug 5 14:18:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:18:12.353897 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3700a5-f8ca-45c5-9b88-420c5413675f None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:12.354185 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3700a5-f8ca-45c5-9b88-420c5413675f None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:12.355581 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3700a5-f8ca-45c5-9b88-420c5413675f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:12.437689 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-9f3700a5-f8ca-45c5-9b88-420c5413675f None None] MFA Rules not processed for user `88d6a1bae9834e2e91d4ab3a4c00959b`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:18:12.438941 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-9f3700a5-f8ca-45c5-9b88-420c5413675f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:12.485851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:12.519935 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 673/1372] 10.222.0.26 () {58 vars in 1061 bytes} [Fri Aug 5 14:18:12 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:18:12.527683 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b3039-0a1c-4828-9ed2-3320480a3170 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:18:12.529246 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-539b3039-0a1c-4828-9ed2-3320480a3170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:12.583729 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:12.639739 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b3039-0a1c-4828-9ed2-3320480a3170 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:18:12.715635 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48087ec-7561-4e18-acbc-3d32535fe58e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:18:12.716255 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48087ec-7561-4e18-acbc-3d32535fe58e service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:12.716580 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48087ec-7561-4e18-acbc-3d32535fe58e service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:12.716820 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48087ec-7561-4e18-acbc-3d32535fe58e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:12.719119 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-e48087ec-7561-4e18-acbc-3d32535fe58e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:12.825042 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:12.840141 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48087ec-7561-4e18-acbc-3d32535fe58e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:18:12.841528 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48087ec-7561-4e18-acbc-3d32535fe58e service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:18:12.859971 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 700/1373] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:18:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:18:13.129549 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936bf42-a72e-4c91-b35a-cb0448d7ba60 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:13.130193 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936bf42-a72e-4c91-b35a-cb0448d7ba60 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:13.130193 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936bf42-a72e-4c91-b35a-cb0448d7ba60 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:13.131825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 674/1374] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:18:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:18:13.136798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f1cf2-7ded-40df-99fc-672c2ba4f0cf None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:13.137089 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f1cf2-7ded-40df-99fc-672c2ba4f0cf None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:13.137500 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f1cf2-7ded-40df-99fc-672c2ba4f0cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:13.206201 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-8c1f1cf2-7ded-40df-99fc-672c2ba4f0cf None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:18:13.207786 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-8c1f1cf2-7ded-40df-99fc-672c2ba4f0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:13.328605 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 701/1375] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:18:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:18:13.372907 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936bf42-a72e-4c91-b35a-cb0448d7ba60 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:18:13.375077 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-9936bf42-a72e-4c91-b35a-cb0448d7ba60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:13.520774 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936bf42-a72e-4c91-b35a-cb0448d7ba60 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:18:13.572650 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:13.633771 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d9efcd-da08-4b6e-82c6-87213dea9cbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:18:13.634379 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9efcd-da08-4b6e-82c6-87213dea9cbf service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:13.634654 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9efcd-da08-4b6e-82c6-87213dea9cbf service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:13.634958 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9efcd-da08-4b6e-82c6-87213dea9cbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:13.636617 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-34d9efcd-da08-4b6e-82c6-87213dea9cbf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:13.726922 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d9efcd-da08-4b6e-82c6-87213dea9cbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:18:13.728555 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d9efcd-da08-4b6e-82c6-87213dea9cbf service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:18:13.747358 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 675/1376] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:18:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:18:15.919571 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53486bd7-1fc9-4a21-8869-f7e5d914e39c None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:15.920115 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53486bd7-1fc9-4a21-8869-f7e5d914e39c None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:15.920308 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53486bd7-1fc9-4a21-8869-f7e5d914e39c None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:15.921705 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 702/1377] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:18:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:18:15.926955 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f90554-93e8-4ca1-9b4a-58b28ea034be None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:15.927390 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f90554-93e8-4ca1-9b4a-58b28ea034be None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:15.927640 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f90554-93e8-4ca1-9b4a-58b28ea034be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:16.025000 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-57f90554-93e8-4ca1-9b4a-58b28ea034be None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:18:16.026238 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-57f90554-93e8-4ca1-9b4a-58b28ea034be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:16.033908 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:16.125016 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 676/1378] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:18:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:18:16.168603 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53486bd7-1fc9-4a21-8869-f7e5d914e39c None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:18:16.170725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-53486bd7-1fc9-4a21-8869-f7e5d914e39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:16.229809 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:16.323301 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53486bd7-1fc9-4a21-8869-f7e5d914e39c None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:18:16.471065 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b5685f-d1c6-45b6-a992-2b8c19191321 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:18:16.472460 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b5685f-d1c6-45b6-a992-2b8c19191321 service nova] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:16.474763 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b5685f-d1c6-45b6-a992-2b8c19191321 service nova] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:16.474763 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b5685f-d1c6-45b6-a992-2b8c19191321 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:16.476756 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-c2b5685f-d1c6-45b6-a992-2b8c19191321 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:16.503077 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:16.602005 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b5685f-d1c6-45b6-a992-2b8c19191321 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:18:16.603598 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b5685f-d1c6-45b6-a992-2b8c19191321 service nova] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:18:16.622692 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 703/1379] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:18:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:18:18.783088 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43374d5b-bde9-4c29-874c-fb5cf2db03af None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:18.783639 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43374d5b-bde9-4c29-874c-fb5cf2db03af None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:18.783639 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43374d5b-bde9-4c29-874c-fb5cf2db03af None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:18.785147 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 677/1380] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:18:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:18:18.790268 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3cdf82-0e30-4629-a840-ddb8976921f3 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:18.790681 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3cdf82-0e30-4629-a840-ddb8976921f3 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:18.790870 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3cdf82-0e30-4629-a840-ddb8976921f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:18.862561 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-7b3cdf82-0e30-4629-a840-ddb8976921f3 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:18:18.863750 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-7b3cdf82-0e30-4629-a840-ddb8976921f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:18.962032 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:18.969093 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 704/1381] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:18:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:18:19.015190 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43374d5b-bde9-4c29-874c-fb5cf2db03af None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:18:19.018344 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-43374d5b-bde9-4c29-874c-fb5cf2db03af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:19.162560 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:19.173353 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43374d5b-bde9-4c29-874c-fb5cf2db03af None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:18:19.303213 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3a3ea3-78ea-4d75-80b6-08fad4dbbfdb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:18:19.304290 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a3ea3-78ea-4d75-80b6-08fad4dbbfdb service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:19.304692 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a3ea3-78ea-4d75-80b6-08fad4dbbfdb service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:19.305060 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a3ea3-78ea-4d75-80b6-08fad4dbbfdb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:19.306922 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ae3a3ea3-78ea-4d75-80b6-08fad4dbbfdb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:19.423504 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3a3ea3-78ea-4d75-80b6-08fad4dbbfdb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:18:19.429824 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3a3ea3-78ea-4d75-80b6-08fad4dbbfdb service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:18:19.442371 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:19.456589 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 678/1382] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:18:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:18:21.648076 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49f150-cb69-45e5-b893-de7bd1a32e2b None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:21.648811 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49f150-cb69-45e5-b893-de7bd1a32e2b None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:21.648811 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49f150-cb69-45e5-b893-de7bd1a32e2b None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:21.650122 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 705/1383] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:18:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:18:21.656161 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33237210-7e15-4720-a9cb-aa50517bf790 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:21.656701 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33237210-7e15-4720-a9cb-aa50517bf790 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:21.656849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33237210-7e15-4720-a9cb-aa50517bf790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:21.725813 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-33237210-7e15-4720-a9cb-aa50517bf790 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:18:21.726913 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-33237210-7e15-4720-a9cb-aa50517bf790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:21.835962 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 679/1384] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:18:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:18:21.880878 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49f150-cb69-45e5-b893-de7bd1a32e2b None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:18:21.882429 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-cf49f150-cb69-45e5-b893-de7bd1a32e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:22.031766 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49f150-cb69-45e5-b893-de7bd1a32e2b None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:18:22.109180 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:22.157181 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600e5e1-9201-4a53-b47c-559528a7aa38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:18:22.157806 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600e5e1-9201-4a53-b47c-559528a7aa38 service nova] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:22.158082 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600e5e1-9201-4a53-b47c-559528a7aa38 service nova] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:22.158359 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600e5e1-9201-4a53-b47c-559528a7aa38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:22.159991 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-c600e5e1-9201-4a53-b47c-559528a7aa38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:22.252271 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c600e5e1-9201-4a53-b47c-559528a7aa38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:18:22.253896 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c600e5e1-9201-4a53-b47c-559528a7aa38 service nova] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:18:22.272957 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 706/1385] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:18:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:18:24.298079 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d043b2-e72f-4340-b7fd-d4c961f0bc63 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:24.298644 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d043b2-e72f-4340-b7fd-d4c961f0bc63 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:24.298644 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d043b2-e72f-4340-b7fd-d4c961f0bc63 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:24.300033 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 680/1386] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:18:24.306463 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b608f-991b-48ab-a051-b8fe00dcc512 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:24.306879 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b608f-991b-48ab-a051-b8fe00dcc512 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:24.307013 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b608f-991b-48ab-a051-b8fe00dcc512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:24.377853 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-5b7b608f-991b-48ab-a051-b8fe00dcc512 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:18:24.379074 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-5b7b608f-991b-48ab-a051-b8fe00dcc512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:24.414040 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:24.481853 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 707/1387] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:18:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:18:24.536235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d043b2-e72f-4340-b7fd-d4c961f0bc63 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:18:24.537793 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-34d043b2-e72f-4340-b7fd-d4c961f0bc63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:24.622314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:24.700434 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d043b2-e72f-4340-b7fd-d4c961f0bc63 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:18:24.821329 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f1d242-388b-430c-83dd-f61c630ad8cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:18:24.822177 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1d242-388b-430c-83dd-f61c630ad8cf service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:24.822517 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1d242-388b-430c-83dd-f61c630ad8cf service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:24.822859 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1d242-388b-430c-83dd-f61c630ad8cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:24.824938 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f8f1d242-388b-430c-83dd-f61c630ad8cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:24.867948 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:24.921440 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f1d242-388b-430c-83dd-f61c630ad8cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:18:24.922988 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f1d242-388b-430c-83dd-f61c630ad8cf service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:18:24.941512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 681/1388] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:18:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:18:27.143692 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21caa6-0f74-4628-b514-5c3cadc4b463 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:27.144290 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21caa6-0f74-4628-b514-5c3cadc4b463 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:27.144290 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21caa6-0f74-4628-b514-5c3cadc4b463 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:27.145606 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 708/1389] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:18:27.150865 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc8e80-0048-439d-b3aa-a690a33e3b25 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:27.151275 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc8e80-0048-439d-b3aa-a690a33e3b25 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:27.151478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc8e80-0048-439d-b3aa-a690a33e3b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:27.220727 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-1fbc8e80-0048-439d-b3aa-a690a33e3b25 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:18:27.221924 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-1fbc8e80-0048-439d-b3aa-a690a33e3b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:27.320302 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 682/1390] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:18:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:18:27.365049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21caa6-0f74-4628-b514-5c3cadc4b463 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:18:27.366666 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-be21caa6-0f74-4628-b514-5c3cadc4b463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:27.513103 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21caa6-0f74-4628-b514-5c3cadc4b463 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:18:27.533344 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:27.629412 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd5859-2f46-42db-acb3-7d65eae5ea76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:18:27.630033 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd5859-2f46-42db-acb3-7d65eae5ea76 service nova] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:27.630312 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd5859-2f46-42db-acb3-7d65eae5ea76 service nova] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:27.630585 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd5859-2f46-42db-acb3-7d65eae5ea76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:27.632235 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-e2bd5859-2f46-42db-acb3-7d65eae5ea76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:27.721782 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd5859-2f46-42db-acb3-7d65eae5ea76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:18:27.723307 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd5859-2f46-42db-acb3-7d65eae5ea76 service nova] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:18:27.741694 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 709/1391] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:18:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:18:28.179070 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76b1cc-181e-4292-b030-712096dd6dd4 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:28.179616 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76b1cc-181e-4292-b030-712096dd6dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:28.179616 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76b1cc-181e-4292-b030-712096dd6dd4 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:28.181072 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 683/1392] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:18:28.186206 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a140f-9529-4cf7-9376-279afb14843f None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:28.186599 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a140f-9529-4cf7-9376-279afb14843f None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:28.186753 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a140f-9529-4cf7-9376-279afb14843f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:28.212364 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:28.253987 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-bd6a140f-9529-4cf7-9376-279afb14843f None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:18:28.255471 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-bd6a140f-9529-4cf7-9376-279afb14843f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:28.352395 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 710/1393] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:18:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:18:28.409660 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a8746-9726-42d7-81ac-13b20d064052 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:28.409940 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a8746-9726-42d7-81ac-13b20d064052 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:28.410213 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a8746-9726-42d7-81ac-13b20d064052 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:28.411536 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 684/1394] 10.222.0.26 () {54 vars in 966 bytes} [Fri Aug 5 14:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:18:28.416572 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f72372-aef9-46fb-bdff-9f4748dce082 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:28.416861 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f72372-aef9-46fb-bdff-9f4748dce082 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:28.417138 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f72372-aef9-46fb-bdff-9f4748dce082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:28.493442 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-77f72372-aef9-46fb-bdff-9f4748dce082 None None] MFA Rules not processed for user `dd1ce2a2b4eb405b999d3fb756795176`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:18:28.494546 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-77f72372-aef9-46fb-bdff-9f4748dce082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:28.538318 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:28.573425 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 711/1395] 10.222.0.26 () {58 vars in 1063 bytes} [Fri Aug 5 14:18:28 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:18:28.581517 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a8746-9726-42d7-81ac-13b20d064052 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:18:28.583095 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-539a8746-9726-42d7-81ac-13b20d064052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:28.597303 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:28.688577 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a8746-9726-42d7-81ac-13b20d064052 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:18:28.761149 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1167dbdd-5db7-4253-adf4-06b1cd92b12e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1ce2a2b4eb405b999d3fb756795176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:18:28.761706 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167dbdd-5db7-4253-adf4-06b1cd92b12e service glance] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:28.761973 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167dbdd-5db7-4253-adf4-06b1cd92b12e service glance] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:28.762234 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167dbdd-5db7-4253-adf4-06b1cd92b12e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:28.763858 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-1167dbdd-5db7-4253-adf4-06b1cd92b12e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:28.829708 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:28.869210 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1167dbdd-5db7-4253-adf4-06b1cd92b12e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:18:28.870658 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1167dbdd-5db7-4253-adf4-06b1cd92b12e service glance] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:18:28.889447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 685/1396] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:18:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:18:29.800845 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b32b04c-6b43-48d4-931f-c41e2a946434 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:29.801388 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b32b04c-6b43-48d4-931f-c41e2a946434 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:29.801388 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b32b04c-6b43-48d4-931f-c41e2a946434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:29.869441 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-1b32b04c-6b43-48d4-931f-c41e2a946434 None None] MFA Rules not processed for user `17e3b509a2de4159bf234594573f5e18`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:18:29.870656 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-1b32b04c-6b43-48d4-931f-c41e2a946434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:29.945793 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b583711-f72c-49bd-9dde-e6082979c2d0 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:29.946375 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b583711-f72c-49bd-9dde-e6082979c2d0 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:29.946375 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b583711-f72c-49bd-9dde-e6082979c2d0 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:29.947729 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 686/1397] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:18:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:18:29.952604 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97915d49-1b3f-41dd-8c54-b3069a20a115 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:29.952954 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97915d49-1b3f-41dd-8c54-b3069a20a115 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:29.953874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97915d49-1b3f-41dd-8c54-b3069a20a115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:29.957541 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 712/1398] 10.222.0.26 () {58 vars in 1009 bytes} [Fri Aug 5 14:18:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:18:30.004765 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ea5b1-5ca4-43fd-9a81-80ec1b29df16 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:30.005048 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ea5b1-5ca4-43fd-9a81-80ec1b29df16 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:30.005321 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ea5b1-5ca4-43fd-9a81-80ec1b29df16 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:30.006803 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 713/1399] 10.222.0.26 () {54 vars in 943 bytes} [Fri Aug 5 14:18:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:18:30.012817 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2ea63-05c6-4b87-abfd-15ae84078764 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:30.013173 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2ea63-05c6-4b87-abfd-15ae84078764 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:30.013548 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2ea63-05c6-4b87-abfd-15ae84078764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:30.023970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-97915d49-1b3f-41dd-8c54-b3069a20a115 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:18:30.025224 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-97915d49-1b3f-41dd-8c54-b3069a20a115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:30.087233 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:30.094948 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-8ce2ea63-05c6-4b87-abfd-15ae84078764 None None] MFA Rules not processed for user `dddd7eea67f34f0cbb5f6a1cd85f1fc5`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:18:30.098824 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-8ce2ea63-05c6-4b87-abfd-15ae84078764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:30.128171 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 687/1400] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:18:30.172788 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97915d49-1b3f-41dd-8c54-b3069a20a115 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:18:30.174038 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-97915d49-1b3f-41dd-8c54-b3069a20a115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:30.200348 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:30.207649 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 714/1401] 10.222.0.26 () {58 vars in 1040 bytes} [Fri Aug 5 14:18:30 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:18:30.217377 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce2ea63-05c6-4b87-abfd-15ae84078764 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:18:30.227561 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce2ea63-05c6-4b87-abfd-15ae84078764 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:18:30.291622 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97915d49-1b3f-41dd-8c54-b3069a20a115 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:18:30.336009 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024def74-7039-4a51-9d12-a4a050ae3e5c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dddd7eea67f34f0cbb5f6a1cd85f1fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:18:30.336587 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024def74-7039-4a51-9d12-a4a050ae3e5c service swift] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:30.336866 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024def74-7039-4a51-9d12-a4a050ae3e5c service swift] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:30.337129 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024def74-7039-4a51-9d12-a4a050ae3e5c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:30.338785 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-024def74-7039-4a51-9d12-a4a050ae3e5c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:30.341365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:30.420134 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee6554-2711-4fff-adfe-d4c0d9d18f2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:18:30.420782 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee6554-2711-4fff-adfe-d4c0d9d18f2a service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:30.421063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee6554-2711-4fff-adfe-d4c0d9d18f2a service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:30.421331 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee6554-2711-4fff-adfe-d4c0d9d18f2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:30.424447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d3ee6554-2711-4fff-adfe-d4c0d9d18f2a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:30.444571 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:30.528255 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ee6554-2711-4fff-adfe-d4c0d9d18f2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:18:30.529898 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ee6554-2711-4fff-adfe-d4c0d9d18f2a service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:18:30.546091 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024def74-7039-4a51-9d12-a4a050ae3e5c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:18:30.547885 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024def74-7039-4a51-9d12-a4a050ae3e5c service swift] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:18:30.555906 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 688/1402] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:18:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 14:18:30.556703 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 715/1403] 10.222.0.26 () {60 vars in 1384 bytes} [Fri Aug 5 14:18:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 05 14:18:32.917186 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156fc2f-aead-43e0-8786-349232340218 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:32.917731 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156fc2f-aead-43e0-8786-349232340218 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:32.917731 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156fc2f-aead-43e0-8786-349232340218 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:32.929099 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 689/1404] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:18:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:18:32.929228 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5974a5-35cc-48b3-974b-cae436ec5e05 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:32.929228 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5974a5-35cc-48b3-974b-cae436ec5e05 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:32.929228 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5974a5-35cc-48b3-974b-cae436ec5e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:33.006614 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-df5974a5-35cc-48b3-974b-cae436ec5e05 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:18:33.023339 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-df5974a5-35cc-48b3-974b-cae436ec5e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:33.129599 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:33.137645 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 716/1405] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:18:33.187411 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156fc2f-aead-43e0-8786-349232340218 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:18:33.189272 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8156fc2f-aead-43e0-8786-349232340218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:33.379963 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:33.382815 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156fc2f-aead-43e0-8786-349232340218 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:18:33.509479 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d0342d-22f5-47d1-aa58-5a97f95f3c09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:18:33.510205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0342d-22f5-47d1-aa58-5a97f95f3c09 service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:33.510543 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0342d-22f5-47d1-aa58-5a97f95f3c09 service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:33.510878 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0342d-22f5-47d1-aa58-5a97f95f3c09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:33.512589 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-06d0342d-22f5-47d1-aa58-5a97f95f3c09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:33.618942 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d0342d-22f5-47d1-aa58-5a97f95f3c09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:18:33.620716 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d0342d-22f5-47d1-aa58-5a97f95f3c09 service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:18:33.642445 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 690/1406] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:18:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:18:35.868950 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8794ab-1962-4591-8168-3086ffe9f214 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:35.869763 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8794ab-1962-4591-8168-3086ffe9f214 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:35.869915 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8794ab-1962-4591-8168-3086ffe9f214 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:35.871752 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 717/1407] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:18:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:18:35.877811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520943e3-c26e-4aa8-b0fd-6b2392a59fd1 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:35.880959 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520943e3-c26e-4aa8-b0fd-6b2392a59fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:35.880959 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520943e3-c26e-4aa8-b0fd-6b2392a59fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:35.883783 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:35.955898 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-520943e3-c26e-4aa8-b0fd-6b2392a59fd1 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:18:35.958094 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-520943e3-c26e-4aa8-b0fd-6b2392a59fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:36.066957 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 691/1408] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:18:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:18:36.112686 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8794ab-1962-4591-8168-3086ffe9f214 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:18:36.114283 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-6a8794ab-1962-4591-8168-3086ffe9f214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:36.280308 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8794ab-1962-4591-8168-3086ffe9f214 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:18:36.357683 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:36.414176 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee92612a-17aa-44a1-b7af-cb762e034872 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:18:36.414794 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92612a-17aa-44a1-b7af-cb762e034872 service nova] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:36.415072 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92612a-17aa-44a1-b7af-cb762e034872 service nova] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:36.415340 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92612a-17aa-44a1-b7af-cb762e034872 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:36.417120 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-ee92612a-17aa-44a1-b7af-cb762e034872 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:36.512963 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee92612a-17aa-44a1-b7af-cb762e034872 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:18:36.518401 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee92612a-17aa-44a1-b7af-cb762e034872 service nova] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:18:36.538679 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 718/1409] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:18:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:18:46.064559 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=64 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:46.535830 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=64 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:54.775435 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3392e3-edce-4d66-92b8-2784d03b8659 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:54.775979 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3392e3-edce-4d66-92b8-2784d03b8659 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:54.775979 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3392e3-edce-4d66-92b8-2784d03b8659 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:54.777275 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 692/1410] 10.222.0.26 () {54 vars in 918 bytes} [Fri Aug 5 14:18:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:18:54.782453 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b03f76-859a-4389-9d23-a2cf478321d2 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:54.782882 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b03f76-859a-4389-9d23-a2cf478321d2 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:54.782999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b03f76-859a-4389-9d23-a2cf478321d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:54.857145 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-f0b03f76-859a-4389-9d23-a2cf478321d2 None None] MFA Rules not processed for user `fff761f6f81a46098f94bd2ec2be9c1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:18:54.859158 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-f0b03f76-859a-4389-9d23-a2cf478321d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:54.962958 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 719/1411] 10.222.0.26 () {58 vars in 1015 bytes} [Fri Aug 5 14:18:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:18:54.977644 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef93fdb-1a0e-4e78-ba0d-8af2522c6beb None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:54.977924 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef93fdb-1a0e-4e78-ba0d-8af2522c6beb None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:54.978195 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef93fdb-1a0e-4e78-ba0d-8af2522c6beb None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:54.979536 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 693/1412] 10.222.0.26 () {54 vars in 918 bytes} [Fri Aug 5 14:18:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:18:54.984805 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6d798-a006-47dd-a810-2ea335c26147 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:54.985084 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6d798-a006-47dd-a810-2ea335c26147 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:54.985352 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6d798-a006-47dd-a810-2ea335c26147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:55.056549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:55.059372 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-54e6d798-a006-47dd-a810-2ea335c26147 None None] MFA Rules not processed for user `5ec4940c99fc4fa8a5bc7cdabc4558fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:18:55.060862 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-54e6d798-a006-47dd-a810-2ea335c26147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:55.174822 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 720/1413] 10.222.0.26 () {58 vars in 1015 bytes} [Fri Aug 5 14:18:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:18:57.641654 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9761b3-9420-4461-870f-fd5eda072a05 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:57.642212 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9761b3-9420-4461-870f-fd5eda072a05 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:57.642212 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9761b3-9420-4461-870f-fd5eda072a05 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:57.643532 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 694/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 14:18:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:18:57.648935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6fde95-f1e5-4622-ad2f-4bf98fa627e2 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:57.649335 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6fde95-f1e5-4622-ad2f-4bf98fa627e2 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:57.649486 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6fde95-f1e5-4622-ad2f-4bf98fa627e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:57.714835 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-1c6fde95-f1e5-4622-ad2f-4bf98fa627e2 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:18:57.716035 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-1c6fde95-f1e5-4622-ad2f-4bf98fa627e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:57.771228 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:57.812148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 721/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 14:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:18:57.856261 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9761b3-9420-4461-870f-fd5eda072a05 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:18:57.858100 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ba9761b3-9420-4461-870f-fd5eda072a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:58.001435 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9761b3-9420-4461-870f-fd5eda072a05 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:18:58.071591 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:58.112982 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b8ea4d-14f2-4680-9b4a-c3654acdfc08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:18:58.113599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8ea4d-14f2-4680-9b4a-c3654acdfc08 service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:58.113875 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8ea4d-14f2-4680-9b4a-c3654acdfc08 service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:58.114142 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8ea4d-14f2-4680-9b4a-c3654acdfc08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:58.115950 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-63b8ea4d-14f2-4680-9b4a-c3654acdfc08 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:58.203648 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b8ea4d-14f2-4680-9b4a-c3654acdfc08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:18:58.205248 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b8ea4d-14f2-4680-9b4a-c3654acdfc08 service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:18:58.223165 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 695/1416] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:18:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:18:59.122462 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767c32b-fb3d-45fd-a9fd-66f665dff299 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:59.123031 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767c32b-fb3d-45fd-a9fd-66f665dff299 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:59.123031 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767c32b-fb3d-45fd-a9fd-66f665dff299 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:59.125197 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 722/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 14:18:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:18:59.130079 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec30e34-87f5-447d-b9ab-8951aaf708d4 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:59.130486 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec30e34-87f5-447d-b9ab-8951aaf708d4 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:59.130624 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec30e34-87f5-447d-b9ab-8951aaf708d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:59.205666 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-0ec30e34-87f5-447d-b9ab-8951aaf708d4 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:18:59.206867 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-0ec30e34-87f5-447d-b9ab-8951aaf708d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:59.232915 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:59.308229 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 696/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 14:18:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:18:59.460287 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8767c32b-fb3d-45fd-a9fd-66f665dff299 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:18:59.461881 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-8767c32b-fb3d-45fd-a9fd-66f665dff299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:59.605509 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8767c32b-fb3d-45fd-a9fd-66f665dff299 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:18:59.645031 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:18:59.715570 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d6de53-633a-48c8-a57c-fb281fb1a6e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:18:59.716104 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d6de53-633a-48c8-a57c-fb281fb1a6e0 service nova] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:18:59.716376 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d6de53-633a-48c8-a57c-fb281fb1a6e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:18:59.716673 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d6de53-633a-48c8-a57c-fb281fb1a6e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:18:59.718431 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-c3d6de53-633a-48c8-a57c-fb281fb1a6e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:18:59.805820 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d6de53-633a-48c8-a57c-fb281fb1a6e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:18:59.807416 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d6de53-633a-48c8-a57c-fb281fb1a6e0 service nova] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:18:59.825362 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 723/1419] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:18:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:19:00.712474 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7232e-456e-485a-8cf0-e6ba0e82535c None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:00.713028 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7232e-456e-485a-8cf0-e6ba0e82535c None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:00.713028 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7232e-456e-485a-8cf0-e6ba0e82535c None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:00.714359 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 697/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 14:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:19:00.719491 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1af299-a1ce-4c02-98f9-2550bcd2b32b None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:00.719922 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1af299-a1ce-4c02-98f9-2550bcd2b32b None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:00.720019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1af299-a1ce-4c02-98f9-2550bcd2b32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:00.773817 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:00.796316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-ea1af299-a1ce-4c02-98f9-2550bcd2b32b None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:19:00.796527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-ea1af299-a1ce-4c02-98f9-2550bcd2b32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:00.890431 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 724/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 14:19:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:19:01.043046 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a7232e-456e-485a-8cf0-e6ba0e82535c None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:19:01.044656 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a2a7232e-456e-485a-8cf0-e6ba0e82535c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:01.189166 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a7232e-456e-485a-8cf0-e6ba0e82535c None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:19:01.200898 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:01.306414 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb206f0e-b01a-4f3b-982a-1f04b4aab19e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:19:01.307024 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb206f0e-b01a-4f3b-982a-1f04b4aab19e service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:01.307297 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb206f0e-b01a-4f3b-982a-1f04b4aab19e service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:01.307565 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb206f0e-b01a-4f3b-982a-1f04b4aab19e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:01.309244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-fb206f0e-b01a-4f3b-982a-1f04b4aab19e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:01.397411 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb206f0e-b01a-4f3b-982a-1f04b4aab19e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:19:01.399062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb206f0e-b01a-4f3b-982a-1f04b4aab19e service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:19:01.417454 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 698/1422] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:19:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:19:02.273417 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649d37b-b150-4dac-86f3-ece9800668f4 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:02.273990 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649d37b-b150-4dac-86f3-ece9800668f4 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:02.273990 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649d37b-b150-4dac-86f3-ece9800668f4 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:02.275295 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 725/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 14:19:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:19:02.280332 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf84865-fbe7-481f-948a-82011bf37291 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:02.280817 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf84865-fbe7-481f-948a-82011bf37291 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:02.281002 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf84865-fbe7-481f-948a-82011bf37291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:02.301141 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:02.354943 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-eaf84865-fbe7-481f-948a-82011bf37291 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:19:02.356143 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-eaf84865-fbe7-481f-948a-82011bf37291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:02.455711 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 699/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 14:19:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:19:02.606876 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649d37b-b150-4dac-86f3-ece9800668f4 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:19:02.609259 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-c649d37b-b150-4dac-86f3-ece9800668f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:02.716346 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:02.753654 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649d37b-b150-4dac-86f3-ece9800668f4 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:19:02.865495 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71631d66-1ae5-4c02-89b5-07e4c9021daf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:19:02.866106 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71631d66-1ae5-4c02-89b5-07e4c9021daf service nova] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:02.866379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71631d66-1ae5-4c02-89b5-07e4c9021daf service nova] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:02.866644 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71631d66-1ae5-4c02-89b5-07e4c9021daf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:02.868235 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-71631d66-1ae5-4c02-89b5-07e4c9021daf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:02.941734 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:02.955161 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71631d66-1ae5-4c02-89b5-07e4c9021daf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:19:02.956760 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71631d66-1ae5-4c02-89b5-07e4c9021daf service nova] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:19:02.975515 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 726/1425] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:19:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:19:03.837838 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6260926-7ac9-4126-bb4f-f3841e51e307 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:03.838357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6260926-7ac9-4126-bb4f-f3841e51e307 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:03.838357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6260926-7ac9-4126-bb4f-f3841e51e307 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:03.839675 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 700/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 14:19:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:19:03.844826 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953806b8-8f81-4dff-8743-6c65f8a0dd67 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:03.845262 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953806b8-8f81-4dff-8743-6c65f8a0dd67 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:03.845378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953806b8-8f81-4dff-8743-6c65f8a0dd67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:03.921937 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-953806b8-8f81-4dff-8743-6c65f8a0dd67 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:19:03.923140 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-953806b8-8f81-4dff-8743-6c65f8a0dd67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:04.017480 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 727/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 14:19:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:19:04.168246 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6260926-7ac9-4126-bb4f-f3841e51e307 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:19:04.169777 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d6260926-7ac9-4126-bb4f-f3841e51e307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:04.253324 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:04.315395 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6260926-7ac9-4126-bb4f-f3841e51e307 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:19:04.428646 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4cf5eb-14ff-421a-9de7-b07d0973a2ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:19:04.429163 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4cf5eb-14ff-421a-9de7-b07d0973a2ca service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:04.429442 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4cf5eb-14ff-421a-9de7-b07d0973a2ca service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:04.429714 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4cf5eb-14ff-421a-9de7-b07d0973a2ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:04.431343 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-1c4cf5eb-14ff-421a-9de7-b07d0973a2ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:04.481040 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:04.529757 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4cf5eb-14ff-421a-9de7-b07d0973a2ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:19:04.532367 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4cf5eb-14ff-421a-9de7-b07d0973a2ca service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:19:04.551849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 701/1428] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:19:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:19:05.390743 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab88a2-c7aa-4db2-839b-190a5b656168 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:05.391326 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab88a2-c7aa-4db2-839b-190a5b656168 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:05.391326 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab88a2-c7aa-4db2-839b-190a5b656168 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:05.392684 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 728/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 14:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:19:05.397639 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90151c7e-fc8c-4b9e-bc09-8f011af03ea3 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:05.398106 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90151c7e-fc8c-4b9e-bc09-8f011af03ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:05.398211 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90151c7e-fc8c-4b9e-bc09-8f011af03ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:05.473280 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-90151c7e-fc8c-4b9e-bc09-8f011af03ea3 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:19:05.474490 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-90151c7e-fc8c-4b9e-bc09-8f011af03ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:05.570814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 702/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 14:19:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:19:05.723100 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cab88a2-c7aa-4db2-839b-190a5b656168 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:19:05.724738 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-9cab88a2-c7aa-4db2-839b-190a5b656168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:05.773113 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:05.868263 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cab88a2-c7aa-4db2-839b-190a5b656168 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:19:05.978092 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed876ee7-fc6a-4137-a265-d0e1105c67f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:19:05.978634 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed876ee7-fc6a-4137-a265-d0e1105c67f2 service nova] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:05.978906 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed876ee7-fc6a-4137-a265-d0e1105c67f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:05.979178 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed876ee7-fc6a-4137-a265-d0e1105c67f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:05.980820 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-ed876ee7-fc6a-4137-a265-d0e1105c67f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:05.988792 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:06.069881 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed876ee7-fc6a-4137-a265-d0e1105c67f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:19:06.071486 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed876ee7-fc6a-4137-a265-d0e1105c67f2 service nova] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:19:06.089123 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 729/1431] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:19:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:19:06.987074 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47967c-e80c-4edf-8b58-fa3b65ab62a8 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:06.987651 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47967c-e80c-4edf-8b58-fa3b65ab62a8 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:06.987651 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47967c-e80c-4edf-8b58-fa3b65ab62a8 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:06.988984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 703/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 14:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:19:06.993846 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b2374-b35d-4222-85e1-dcaa45d06ad3 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:06.994302 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b2374-b35d-4222-85e1-dcaa45d06ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:06.994436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b2374-b35d-4222-85e1-dcaa45d06ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:07.072078 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-aa7b2374-b35d-4222-85e1-dcaa45d06ad3 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:19:07.073314 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-aa7b2374-b35d-4222-85e1-dcaa45d06ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:07.140045 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:07.168904 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 730/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 14:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:19:07.320255 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e47967c-e80c-4edf-8b58-fa3b65ab62a8 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:19:07.321707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8e47967c-e80c-4edf-8b58-fa3b65ab62a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:07.340204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:07.466735 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e47967c-e80c-4edf-8b58-fa3b65ab62a8 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:19:07.552567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:07.577954 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588e1382-f51b-48cb-a468-ef836c64c9e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:19:07.578492 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e1382-f51b-48cb-a468-ef836c64c9e0 service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:07.578762 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e1382-f51b-48cb-a468-ef836c64c9e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:07.579023 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e1382-f51b-48cb-a468-ef836c64c9e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:07.580668 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-588e1382-f51b-48cb-a468-ef836c64c9e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:07.667074 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588e1382-f51b-48cb-a468-ef836c64c9e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:19:07.668675 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588e1382-f51b-48cb-a468-ef836c64c9e0 service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:19:07.686451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 704/1434] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:19:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:19:08.570204 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54494e-0c01-4ee9-9dd6-36e415ca6d26 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:08.570694 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54494e-0c01-4ee9-9dd6-36e415ca6d26 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:08.570807 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54494e-0c01-4ee9-9dd6-36e415ca6d26 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:08.572121 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 731/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 14:19:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:19:08.577702 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0becd9b4-4ead-4f37-8b98-8e87b4351f21 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:08.578111 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0becd9b4-4ead-4f37-8b98-8e87b4351f21 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:08.578329 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0becd9b4-4ead-4f37-8b98-8e87b4351f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:08.653977 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-0becd9b4-4ead-4f37-8b98-8e87b4351f21 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:19:08.655494 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-0becd9b4-4ead-4f37-8b98-8e87b4351f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:08.697284 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:08.751896 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 705/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 14:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:19:08.904265 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54494e-0c01-4ee9-9dd6-36e415ca6d26 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:19:08.905931 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-5a54494e-0c01-4ee9-9dd6-36e415ca6d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:09.050550 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54494e-0c01-4ee9-9dd6-36e415ca6d26 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:19:09.103660 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:09.162675 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b959b4fb-821c-40ea-96b0-ef6ae07ded26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:19:09.163208 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959b4fb-821c-40ea-96b0-ef6ae07ded26 service nova] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:09.163482 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959b4fb-821c-40ea-96b0-ef6ae07ded26 service nova] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:09.163748 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959b4fb-821c-40ea-96b0-ef6ae07ded26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:09.165376 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-b959b4fb-821c-40ea-96b0-ef6ae07ded26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:09.251850 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b959b4fb-821c-40ea-96b0-ef6ae07ded26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:19:09.253437 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b959b4fb-821c-40ea-96b0-ef6ae07ded26 service nova] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:19:09.271459 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 732/1437] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:19:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:19:10.136787 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a781d-1c54-4a76-ab35-b831e52b0512 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:10.137346 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a781d-1c54-4a76-ab35-b831e52b0512 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:10.137346 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a781d-1c54-4a76-ab35-b831e52b0512 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:10.138650 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 706/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 14:19:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:19:10.143732 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b60b5-dee0-48cd-8220-04230ebc6010 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:10.144166 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b60b5-dee0-48cd-8220-04230ebc6010 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:10.144306 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b60b5-dee0-48cd-8220-04230ebc6010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:10.220522 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-5c4b60b5-dee0-48cd-8220-04230ebc6010 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:19:10.221745 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-5c4b60b5-dee0-48cd-8220-04230ebc6010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:10.223752 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:10.318760 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 733/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 14:19:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:19:10.469315 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a781d-1c54-4a76-ab35-b831e52b0512 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:19:10.470774 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-139a781d-1c54-4a76-ab35-b831e52b0512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:10.614288 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a781d-1c54-4a76-ab35-b831e52b0512 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:19:10.640869 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:10.730257 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c895df35-87de-4934-a14c-1e1ff6ae00a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:19:10.730798 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c895df35-87de-4934-a14c-1e1ff6ae00a5 service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:10.731074 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c895df35-87de-4934-a14c-1e1ff6ae00a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:10.731344 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c895df35-87de-4934-a14c-1e1ff6ae00a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:10.734019 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c895df35-87de-4934-a14c-1e1ff6ae00a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:10.827517 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c895df35-87de-4934-a14c-1e1ff6ae00a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:19:10.829437 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c895df35-87de-4934-a14c-1e1ff6ae00a5 service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:19:10.847746 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 707/1440] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:19:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:19:12.777392 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb768c4-4fde-4e4b-809d-1a00d80f7c2c None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:12.777906 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb768c4-4fde-4e4b-809d-1a00d80f7c2c None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:12.777906 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb768c4-4fde-4e4b-809d-1a00d80f7c2c None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:12.779260 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 734/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 14:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:19:12.784912 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d406b81-1828-4658-899b-04269c1a20aa None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:12.785331 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d406b81-1828-4658-899b-04269c1a20aa None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:12.785451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d406b81-1828-4658-899b-04269c1a20aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:12.828165 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:12.851941 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-8d406b81-1828-4658-899b-04269c1a20aa None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:19:12.853252 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8d406b81-1828-4658-899b-04269c1a20aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:12.948972 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 708/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 14:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:19:12.987975 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb768c4-4fde-4e4b-809d-1a00d80f7c2c None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:19:12.989586 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-afb768c4-4fde-4e4b-809d-1a00d80f7c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:13.089708 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb768c4-4fde-4e4b-809d-1a00d80f7c2c None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:19:13.122276 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:13.163575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef5e0e-fa2d-4024-8ca1-a1d6c45aba9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:19:13.164189 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef5e0e-fa2d-4024-8ca1-a1d6c45aba9c service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:13.164503 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef5e0e-fa2d-4024-8ca1-a1d6c45aba9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:13.164780 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef5e0e-fa2d-4024-8ca1-a1d6c45aba9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:13.166399 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-c2ef5e0e-fa2d-4024-8ca1-a1d6c45aba9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:13.274989 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef5e0e-fa2d-4024-8ca1-a1d6c45aba9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:19:13.276438 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef5e0e-fa2d-4024-8ca1-a1d6c45aba9c service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:19:13.295138 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 735/1443] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:19:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:19:15.470271 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99222d-0b91-44a9-ba65-6de95bbcd830 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:15.470749 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99222d-0b91-44a9-ba65-6de95bbcd830 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:15.470861 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99222d-0b91-44a9-ba65-6de95bbcd830 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:15.472212 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 709/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 14:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:19:15.477368 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2ca7c-a8e0-4eeb-a33f-7e5d3a364d92 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:15.477792 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2ca7c-a8e0-4eeb-a33f-7e5d3a364d92 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:15.477940 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2ca7c-a8e0-4eeb-a33f-7e5d3a364d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:15.544038 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-baa2ca7c-a8e0-4eeb-a33f-7e5d3a364d92 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:19:15.545958 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-baa2ca7c-a8e0-4eeb-a33f-7e5d3a364d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:15.558472 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:15.642300 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 736/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 14:19:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:19:15.681684 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c99222d-0b91-44a9-ba65-6de95bbcd830 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:19:15.684005 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-2c99222d-0b91-44a9-ba65-6de95bbcd830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:15.786135 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c99222d-0b91-44a9-ba65-6de95bbcd830 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:19:15.797880 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:15.858290 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765e333-a164-46b8-abbd-924870a6ef39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:19:15.858905 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765e333-a164-46b8-abbd-924870a6ef39 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:15.859184 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765e333-a164-46b8-abbd-924870a6ef39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:15.859457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765e333-a164-46b8-abbd-924870a6ef39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:15.861134 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8765e333-a164-46b8-abbd-924870a6ef39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:15.967986 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8765e333-a164-46b8-abbd-924870a6ef39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:19:15.969452 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8765e333-a164-46b8-abbd-924870a6ef39 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:19:15.987915 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 710/1446] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:19:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:19:18.361654 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ef6d6-c442-43d5-8165-51fb9636eeb5 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:18.362229 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ef6d6-c442-43d5-8165-51fb9636eeb5 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:18.362229 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ef6d6-c442-43d5-8165-51fb9636eeb5 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:18.363668 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 737/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 14:19:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:19:18.368983 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89847c34-5be0-4619-afc3-e0c0cca2e745 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:18.369383 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89847c34-5be0-4619-afc3-e0c0cca2e745 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:18.369557 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89847c34-5be0-4619-afc3-e0c0cca2e745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:18.417879 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:18.435276 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-89847c34-5be0-4619-afc3-e0c0cca2e745 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:19:18.437201 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-89847c34-5be0-4619-afc3-e0c0cca2e745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:18.533418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 711/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 14:19:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:19:18.572525 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033ef6d6-c442-43d5-8165-51fb9636eeb5 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:19:18.574106 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-033ef6d6-c442-43d5-8165-51fb9636eeb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:18.675262 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033ef6d6-c442-43d5-8165-51fb9636eeb5 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:19:18.714061 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:18.747774 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e6647-65e3-4419-b872-b45e0bb95253 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:19:18.748564 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e6647-65e3-4419-b872-b45e0bb95253 service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:18.749074 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e6647-65e3-4419-b872-b45e0bb95253 service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:18.749553 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e6647-65e3-4419-b872-b45e0bb95253 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:18.751566 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-268e6647-65e3-4419-b872-b45e0bb95253 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:18.858099 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268e6647-65e3-4419-b872-b45e0bb95253 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:19:18.859522 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268e6647-65e3-4419-b872-b45e0bb95253 service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:19:18.879058 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 738/1449] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:19:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 14:19:21.004601 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5786a8de-03ca-419c-9b12-a55418443b1c None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:21.005147 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5786a8de-03ca-419c-9b12-a55418443b1c None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:21.017561 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5786a8de-03ca-419c-9b12-a55418443b1c None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:21.017561 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 712/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 14:19:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:19:21.017792 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9bb5ec-7847-479f-aaa9-af89a8c0af70 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:21.017792 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9bb5ec-7847-479f-aaa9-af89a8c0af70 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:21.017792 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9bb5ec-7847-479f-aaa9-af89a8c0af70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:21.078477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-8a9bb5ec-7847-479f-aaa9-af89a8c0af70 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:19:21.079662 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-8a9bb5ec-7847-479f-aaa9-af89a8c0af70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:21.099673 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:21.175893 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 739/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 14:19:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:19:21.215025 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5786a8de-03ca-419c-9b12-a55418443b1c None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:19:21.216567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5786a8de-03ca-419c-9b12-a55418443b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:21.319890 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5786a8de-03ca-419c-9b12-a55418443b1c None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:19:21.338833 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:21.393210 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2366f981-c8e7-406c-b52e-8e1905beb5e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:19:21.393821 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366f981-c8e7-406c-b52e-8e1905beb5e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:21.394102 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366f981-c8e7-406c-b52e-8e1905beb5e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:21.394370 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366f981-c8e7-406c-b52e-8e1905beb5e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:21.396136 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-2366f981-c8e7-406c-b52e-8e1905beb5e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:21.506597 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2366f981-c8e7-406c-b52e-8e1905beb5e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:19:21.508027 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2366f981-c8e7-406c-b52e-8e1905beb5e4 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:19:21.526479 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 713/1452] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:19:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:19:23.646277 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537929b-55ae-4fea-ab70-c72f7faa32cc None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:23.646836 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537929b-55ae-4fea-ab70-c72f7faa32cc None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:23.646836 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537929b-55ae-4fea-ab70-c72f7faa32cc None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:23.648161 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 740/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 14:19:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:19:23.653345 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d634d-f78b-4c31-9f36-1fa9cb1f8f9f None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:23.653756 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d634d-f78b-4c31-9f36-1fa9cb1f8f9f None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:23.653900 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d634d-f78b-4c31-9f36-1fa9cb1f8f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:23.716472 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:23.719373 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-2d4d634d-f78b-4c31-9f36-1fa9cb1f8f9f None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:19:23.722421 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-2d4d634d-f78b-4c31-9f36-1fa9cb1f8f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:23.816476 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 714/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 14:19:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:19:23.854627 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537929b-55ae-4fea-ab70-c72f7faa32cc None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:19:23.856145 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-a537929b-55ae-4fea-ab70-c72f7faa32cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:23.958652 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537929b-55ae-4fea-ab70-c72f7faa32cc None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:19:24.007323 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:24.031783 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c69c7e-b2eb-4158-962a-155eb77edc2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:19:24.032399 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c69c7e-b2eb-4158-962a-155eb77edc2f service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:24.032709 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c69c7e-b2eb-4158-962a-155eb77edc2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:24.032994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c69c7e-b2eb-4158-962a-155eb77edc2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:24.034682 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-79c69c7e-b2eb-4158-962a-155eb77edc2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:24.142574 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c69c7e-b2eb-4158-962a-155eb77edc2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:19:24.144013 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c69c7e-b2eb-4158-962a-155eb77edc2f service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:19:24.162065 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 741/1455] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:19:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:19:26.220224 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee2c35-97d2-4360-a345-21b347d946cb None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:26.220837 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee2c35-97d2-4360-a345-21b347d946cb None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:26.220837 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee2c35-97d2-4360-a345-21b347d946cb None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:26.222151 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 715/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 14:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:19:26.227338 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df05ae3-eaf6-4543-bba4-24f28fe94641 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:26.227740 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df05ae3-eaf6-4543-bba4-24f28fe94641 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:26.227886 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df05ae3-eaf6-4543-bba4-24f28fe94641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:26.294135 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-7df05ae3-eaf6-4543-bba4-24f28fe94641 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:19:26.295407 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-7df05ae3-eaf6-4543-bba4-24f28fe94641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:26.324886 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:26.393382 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 742/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 14:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:19:26.432046 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee2c35-97d2-4360-a345-21b347d946cb None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:19:26.433628 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e4ee2c35-97d2-4360-a345-21b347d946cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:26.537413 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee2c35-97d2-4360-a345-21b347d946cb None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:19:26.565778 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:26.610544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f05675-e2a6-4358-80f9-4137f940f720 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:19:26.611153 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f05675-e2a6-4358-80f9-4137f940f720 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:26.611430 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f05675-e2a6-4358-80f9-4137f940f720 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:26.611701 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f05675-e2a6-4358-80f9-4137f940f720 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:26.613357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-00f05675-e2a6-4358-80f9-4137f940f720 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:26.719238 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f05675-e2a6-4358-80f9-4137f940f720 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:19:26.720730 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f05675-e2a6-4358-80f9-4137f940f720 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:19:26.738948 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 716/1458] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:19:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:19:28.828583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96405757-cf11-46c7-ae1c-a14fff5e715b None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:28.829126 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96405757-cf11-46c7-ae1c-a14fff5e715b None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:28.829126 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96405757-cf11-46c7-ae1c-a14fff5e715b None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:28.830443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 743/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 14:19:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:19:28.835702 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10a9cc-aa4e-4e0c-9f75-44f017301f0c None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:28.836131 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10a9cc-aa4e-4e0c-9f75-44f017301f0c None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:28.836239 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10a9cc-aa4e-4e0c-9f75-44f017301f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:28.900301 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-8e10a9cc-aa4e-4e0c-9f75-44f017301f0c None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:19:28.901522 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8e10a9cc-aa4e-4e0c-9f75-44f017301f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:28.910255 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:28.996872 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 717/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 14:19:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:19:29.034644 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96405757-cf11-46c7-ae1c-a14fff5e715b None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:19:29.038863 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-96405757-cf11-46c7-ae1c-a14fff5e715b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:29.140573 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96405757-cf11-46c7-ae1c-a14fff5e715b None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:19:29.195505 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:29.214285 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea871a-9fac-4482-9ed3-1b2b2b678889 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:19:29.214889 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea871a-9fac-4482-9ed3-1b2b2b678889 service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:29.215162 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea871a-9fac-4482-9ed3-1b2b2b678889 service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:29.215428 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea871a-9fac-4482-9ed3-1b2b2b678889 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:29.217987 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-77ea871a-9fac-4482-9ed3-1b2b2b678889 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:29.327455 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea871a-9fac-4482-9ed3-1b2b2b678889 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:19:29.328923 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea871a-9fac-4482-9ed3-1b2b2b678889 service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:19:29.347862 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 744/1461] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:19:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:19:31.391311 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be984ce-bd7e-461c-8e28-d45e5bc4d75e None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:31.391888 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be984ce-bd7e-461c-8e28-d45e5bc4d75e None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:31.391888 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be984ce-bd7e-461c-8e28-d45e5bc4d75e None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:31.393225 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 718/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 14:19:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:19:31.398654 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e6cee9-aedc-4fb2-8afa-3f66e53b1e7f None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:31.399071 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e6cee9-aedc-4fb2-8afa-3f66e53b1e7f None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:31.399195 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e6cee9-aedc-4fb2-8afa-3f66e53b1e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:31.467963 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-76e6cee9-aedc-4fb2-8afa-3f66e53b1e7f None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:19:31.469224 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-76e6cee9-aedc-4fb2-8afa-3f66e53b1e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:31.507933 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:31.568035 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 745/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 14:19:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:19:31.606401 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be984ce-bd7e-461c-8e28-d45e5bc4d75e None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:19:31.609059 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-9be984ce-bd7e-461c-8e28-d45e5bc4d75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:31.714527 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be984ce-bd7e-461c-8e28-d45e5bc4d75e None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:19:31.760694 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:31.800745 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9818e7d-b83d-4074-b8f6-b35f87166f68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:19:31.801369 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9818e7d-b83d-4074-b8f6-b35f87166f68 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:31.801646 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9818e7d-b83d-4074-b8f6-b35f87166f68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:31.801918 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9818e7d-b83d-4074-b8f6-b35f87166f68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:31.803566 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d9818e7d-b83d-4074-b8f6-b35f87166f68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:31.930106 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9818e7d-b83d-4074-b8f6-b35f87166f68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:19:31.931530 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9818e7d-b83d-4074-b8f6-b35f87166f68 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:19:31.952864 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 719/1464] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:19:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:19:34.101112 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edbba1-8bd4-44f1-8a2b-76ef697c7793 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:34.101654 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edbba1-8bd4-44f1-8a2b-76ef697c7793 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:34.101654 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edbba1-8bd4-44f1-8a2b-76ef697c7793 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:34.103037 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 746/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 14:19:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:19:34.109239 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60cc168-877f-4724-88a2-04a85f7eef80 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:34.109646 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60cc168-877f-4724-88a2-04a85f7eef80 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:34.109782 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60cc168-877f-4724-88a2-04a85f7eef80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:34.174820 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-a60cc168-877f-4724-88a2-04a85f7eef80 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:19:34.176024 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a60cc168-877f-4724-88a2-04a85f7eef80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:34.194155 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:34.272293 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 720/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 14:19:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:19:34.308581 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35edbba1-8bd4-44f1-8a2b-76ef697c7793 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:19:34.310746 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-35edbba1-8bd4-44f1-8a2b-76ef697c7793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:34.414486 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35edbba1-8bd4-44f1-8a2b-76ef697c7793 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:19:34.480475 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:34.488685 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e5c52-9c41-4a1a-b8c9-262dcbcea2a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:19:34.489302 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e5c52-9c41-4a1a-b8c9-262dcbcea2a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:34.489578 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e5c52-9c41-4a1a-b8c9-262dcbcea2a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:34.489850 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e5c52-9c41-4a1a-b8c9-262dcbcea2a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:34.491565 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-276e5c52-9c41-4a1a-b8c9-262dcbcea2a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:34.600321 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e5c52-9c41-4a1a-b8c9-262dcbcea2a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:19:34.601779 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e5c52-9c41-4a1a-b8c9-262dcbcea2a6 service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:19:34.620474 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 747/1467] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:19:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:19:36.626299 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283aca95-66b9-4d86-9541-54ff934a0b25 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:36.626892 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283aca95-66b9-4d86-9541-54ff934a0b25 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:36.626892 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283aca95-66b9-4d86-9541-54ff934a0b25 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:36.628204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 721/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 14:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:19:36.633516 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469d8eb-de1d-4be3-8540-35091f61e4fa None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:36.633959 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469d8eb-de1d-4be3-8540-35091f61e4fa None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:36.634096 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469d8eb-de1d-4be3-8540-35091f61e4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:36.698954 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-8469d8eb-de1d-4be3-8540-35091f61e4fa None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:19:36.700157 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-8469d8eb-de1d-4be3-8540-35091f61e4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:36.748117 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:36.796008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 748/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 14:19:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:19:36.832685 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283aca95-66b9-4d86-9541-54ff934a0b25 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:19:36.834203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-283aca95-66b9-4d86-9541-54ff934a0b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:36.936956 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283aca95-66b9-4d86-9541-54ff934a0b25 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:19:36.986822 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:37.012709 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40ec40-8e1f-4c3b-8338-479ffac6843a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:19:37.013413 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40ec40-8e1f-4c3b-8338-479ffac6843a service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:37.013801 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40ec40-8e1f-4c3b-8338-479ffac6843a service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:37.014076 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40ec40-8e1f-4c3b-8338-479ffac6843a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:37.015748 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-be40ec40-8e1f-4c3b-8338-479ffac6843a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:37.122293 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be40ec40-8e1f-4c3b-8338-479ffac6843a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:19:37.124250 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be40ec40-8e1f-4c3b-8338-479ffac6843a service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:19:37.142700 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 722/1470] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:19:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:19:46.794679 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=22 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:47.160456 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=66 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:57.451290 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6d338-e7ab-41a8-b7db-e57e84e83a43 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:57.451863 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6d338-e7ab-41a8-b7db-e57e84e83a43 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:57.451863 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6d338-e7ab-41a8-b7db-e57e84e83a43 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:57.453203 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 749/1471] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:19:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:19:57.458451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bca7ea-3b77-4329-bbb0-e1193dd0ff92 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:57.458886 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bca7ea-3b77-4329-bbb0-e1193dd0ff92 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:57.458987 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bca7ea-3b77-4329-bbb0-e1193dd0ff92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:57.525247 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-c2bca7ea-3b77-4329-bbb0-e1193dd0ff92 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:19:57.526342 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c2bca7ea-3b77-4329-bbb0-e1193dd0ff92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:57.622198 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 723/1472] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:19:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:19:57.704811 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b64ec-1529-4008-b58f-070331fe94de None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:57.705032 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b64ec-1529-4008-b58f-070331fe94de None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:57.705330 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b64ec-1529-4008-b58f-070331fe94de None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:57.706648 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 750/1473] 10.222.0.26 () {54 vars in 966 bytes} [Fri Aug 5 14:19:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:19:57.711290 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dedd859-76aa-4377-85a7-fa2052015ec0 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:57.711577 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dedd859-76aa-4377-85a7-fa2052015ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:57.711850 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dedd859-76aa-4377-85a7-fa2052015ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:57.787305 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-6dedd859-76aa-4377-85a7-fa2052015ec0 None None] MFA Rules not processed for user `dd1ce2a2b4eb405b999d3fb756795176`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:19:57.788535 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-6dedd859-76aa-4377-85a7-fa2052015ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:57.810577 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:57.867406 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 724/1474] 10.222.0.26 () {58 vars in 1063 bytes} [Fri Aug 5 14:19:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:19:57.876101 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6b64ec-1529-4008-b58f-070331fe94de None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:19:57.877691 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-8d6b64ec-1529-4008-b58f-070331fe94de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:57.980485 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6b64ec-1529-4008-b58f-070331fe94de None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:19:58.052382 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bbd77e-a73c-4398-ac56-7df6afd39f68 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1ce2a2b4eb405b999d3fb756795176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:19:58.053015 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bbd77e-a73c-4398-ac56-7df6afd39f68 service glance] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:19:58.053288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bbd77e-a73c-4398-ac56-7df6afd39f68 service glance] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:19:58.053554 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bbd77e-a73c-4398-ac56-7df6afd39f68 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:19:58.055151 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-90bbd77e-a73c-4398-ac56-7df6afd39f68 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:19:58.107816 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:19:58.161254 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bbd77e-a73c-4398-ac56-7df6afd39f68 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:19:58.162661 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bbd77e-a73c-4398-ac56-7df6afd39f68 service glance] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:19:58.180579 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 751/1475] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:19:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:20:00.279984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507642c-5b47-4fe0-b955-aef34e68ad96 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:20:00.280564 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507642c-5b47-4fe0-b955-aef34e68ad96 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:20:00.280564 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507642c-5b47-4fe0-b955-aef34e68ad96 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:20:00.281907 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 725/1476] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:20:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:20:00.287235 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa793ee1-32c8-43f3-b9d2-1139233c6c34 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:20:00.287690 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa793ee1-32c8-43f3-b9d2-1139233c6c34 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:20:00.287800 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa793ee1-32c8-43f3-b9d2-1139233c6c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:20:00.352383 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-aa793ee1-32c8-43f3-b9d2-1139233c6c34 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:20:00.353617 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-aa793ee1-32c8-43f3-b9d2-1139233c6c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:20:00.449882 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 752/1477] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:20:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:20:00.473416 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a507642c-5b47-4fe0-b955-aef34e68ad96 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:20:00.474874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a507642c-5b47-4fe0-b955-aef34e68ad96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:20:00.599572 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a507642c-5b47-4fe0-b955-aef34e68ad96 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:20:00.646966 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:20:00.692109 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f2804-191e-4c03-b91e-926778eb3945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:20:00.692749 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f2804-191e-4c03-b91e-926778eb3945 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:20:00.693027 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f2804-191e-4c03-b91e-926778eb3945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:20:00.693294 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f2804-191e-4c03-b91e-926778eb3945 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:20:00.693983 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0f2804-191e-4c03-b91e-926778eb3945 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:20:00.695716 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: WARNING py.warnings [None req-0a0f2804-191e-4c03-b91e-926778eb3945 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 14:20:00.695716 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: warnings.warn(msg) Aug 05 14:20:00.695716 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: Aug 05 14:20:00.696890 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0f2804-191e-4c03-b91e-926778eb3945 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:20:00.716462 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 726/1478] 10.222.0.26 () {56 vars in 1108 bytes} [Fri Aug 5 14:20:00 2022] GET /identity/v3/projects => generated 2553 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:20:02.764852 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c4340-fbf3-4e58-b0fe-8948da275a71 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:20:02.765413 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c4340-fbf3-4e58-b0fe-8948da275a71 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:20:02.765413 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c4340-fbf3-4e58-b0fe-8948da275a71 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:20:02.766735 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 753/1479] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:20:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:20:02.772103 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758a6c2-2ed3-4c14-8f96-a6a1339466a0 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:20:02.772549 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758a6c2-2ed3-4c14-8f96-a6a1339466a0 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:20:02.772691 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758a6c2-2ed3-4c14-8f96-a6a1339466a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:20:02.837742 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-4758a6c2-2ed3-4c14-8f96-a6a1339466a0 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:20:02.838878 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-4758a6c2-2ed3-4c14-8f96-a6a1339466a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:20:02.934251 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 727/1480] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:20:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:20:02.977490 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c4340-fbf3-4e58-b0fe-8948da275a71 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:20:02.979008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-dd1c4340-fbf3-4e58-b0fe-8948da275a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:20:02.988965 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:20:03.125756 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c4340-fbf3-4e58-b0fe-8948da275a71 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:20:03.206235 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:20:03.240232 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235bf573-99da-408c-ada9-67793e032803 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:20:03.240877 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235bf573-99da-408c-ada9-67793e032803 service nova] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:20:03.241156 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235bf573-99da-408c-ada9-67793e032803 service nova] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:20:03.241428 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235bf573-99da-408c-ada9-67793e032803 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:20:03.243058 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-235bf573-99da-408c-ada9-67793e032803 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:20:03.333237 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235bf573-99da-408c-ada9-67793e032803 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:20:03.334865 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235bf573-99da-408c-ada9-67793e032803 service nova] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:20:03.352829 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 754/1481] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:20:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:20:05.531913 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d4405-9614-4986-8e13-effc097d8bc5 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:20:05.532331 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d4405-9614-4986-8e13-effc097d8bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:20:05.532496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d4405-9614-4986-8e13-effc097d8bc5 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:20:05.533845 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 728/1482] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:20:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:20:05.539086 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557cbed-7722-4a0c-83c6-179251707887 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:20:05.539490 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557cbed-7722-4a0c-83c6-179251707887 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:20:05.539734 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557cbed-7722-4a0c-83c6-179251707887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:20:05.606474 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-4557cbed-7722-4a0c-83c6-179251707887 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:20:05.607683 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-4557cbed-7722-4a0c-83c6-179251707887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:20:05.655690 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:20:05.706175 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 755/1483] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:20:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:20:05.742891 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d4405-9614-4986-8e13-effc097d8bc5 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:20:05.744492 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-889d4405-9614-4986-8e13-effc097d8bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:20:05.768999 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:20:05.848741 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d4405-9614-4986-8e13-effc097d8bc5 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:20:05.920555 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd27986-8d43-4c71-a419-f9f3168596df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:20:05.921164 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd27986-8d43-4c71-a419-f9f3168596df service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:20:05.921499 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd27986-8d43-4c71-a419-f9f3168596df service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:20:05.921703 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd27986-8d43-4c71-a419-f9f3168596df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:20:05.923305 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-1cd27986-8d43-4c71-a419-f9f3168596df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:20:06.006534 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:20:06.035499 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd27986-8d43-4c71-a419-f9f3168596df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:20:06.037041 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd27986-8d43-4c71-a419-f9f3168596df service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:20:06.055796 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 729/1484] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:20:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:20:08.115579 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5cb54-538b-4c78-b56c-910ef5369277 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:20:08.116215 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5cb54-538b-4c78-b56c-910ef5369277 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:20:08.116215 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5cb54-538b-4c78-b56c-910ef5369277 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:20:08.118353 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 756/1485] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:20:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:20:08.124706 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef157a9-3a33-4cd8-8af6-553f8e0440da None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:20:08.125217 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef157a9-3a33-4cd8-8af6-553f8e0440da None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:20:08.125324 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef157a9-3a33-4cd8-8af6-553f8e0440da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:20:08.191141 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-fef157a9-3a33-4cd8-8af6-553f8e0440da None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:20:08.192400 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-fef157a9-3a33-4cd8-8af6-553f8e0440da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:20:08.287483 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 730/1486] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:20:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:20:08.326471 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a5cb54-538b-4c78-b56c-910ef5369277 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:20:08.328038 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-b8a5cb54-538b-4c78-b56c-910ef5369277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:20:08.429306 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a5cb54-538b-4c78-b56c-910ef5369277 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:20:08.499746 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:20:08.501476 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c58f5-ca08-4af0-9d17-7652bc3172de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:20:08.502084 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c58f5-ca08-4af0-9d17-7652bc3172de service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:20:08.502359 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c58f5-ca08-4af0-9d17-7652bc3172de service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:20:08.502623 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c58f5-ca08-4af0-9d17-7652bc3172de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:20:08.504224 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-679c58f5-ca08-4af0-9d17-7652bc3172de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:20:08.613973 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c58f5-ca08-4af0-9d17-7652bc3172de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:20:08.615396 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c58f5-ca08-4af0-9d17-7652bc3172de service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:20:08.633394 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 757/1487] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:20:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:20:10.741230 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d8048-5abf-4f3c-8013-81fb04018ec5 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:20:10.741766 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d8048-5abf-4f3c-8013-81fb04018ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:20:10.741766 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d8048-5abf-4f3c-8013-81fb04018ec5 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:20:10.743130 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 731/1488] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:20:10.748924 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70260cac-529c-4d20-a240-18b1f8830301 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:20:10.749396 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70260cac-529c-4d20-a240-18b1f8830301 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:20:10.749504 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70260cac-529c-4d20-a240-18b1f8830301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:20:10.814641 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-70260cac-529c-4d20-a240-18b1f8830301 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:20:10.815767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-70260cac-529c-4d20-a240-18b1f8830301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:20:10.867468 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:20:10.911381 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 758/1489] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:20:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:20:10.952188 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120d8048-5abf-4f3c-8013-81fb04018ec5 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:20:10.953789 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-120d8048-5abf-4f3c-8013-81fb04018ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:20:10.990004 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:20:11.057922 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120d8048-5abf-4f3c-8013-81fb04018ec5 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:20:11.131207 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a780523-6527-45e9-8cdd-b1add4271634 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:20:11.131819 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a780523-6527-45e9-8cdd-b1add4271634 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:20:11.132093 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a780523-6527-45e9-8cdd-b1add4271634 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:20:11.132361 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a780523-6527-45e9-8cdd-b1add4271634 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:20:11.134015 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5a780523-6527-45e9-8cdd-b1add4271634 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:20:11.212575 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:20:11.240932 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a780523-6527-45e9-8cdd-b1add4271634 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:20:11.242400 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a780523-6527-45e9-8cdd-b1add4271634 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:20:11.260406 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 732/1490] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:20:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:20:13.641797 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286826b7-af77-4289-b061-0192858be98c None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:20:13.642347 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286826b7-af77-4289-b061-0192858be98c None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:20:13.642347 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286826b7-af77-4289-b061-0192858be98c None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:20:13.643681 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 759/1491] 10.222.0.26 () {54 vars in 927 bytes} [Fri Aug 5 14:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:20:13.648900 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070aab7-5e09-465a-ab76-367fb5b57cf4 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:20:13.649356 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070aab7-5e09-465a-ab76-367fb5b57cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:20:13.649462 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070aab7-5e09-465a-ab76-367fb5b57cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:20:13.714489 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-0070aab7-5e09-465a-ab76-367fb5b57cf4 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:20:13.716496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-0070aab7-5e09-465a-ab76-367fb5b57cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:20:13.811152 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 733/1492] 10.222.0.26 () {58 vars in 1024 bytes} [Fri Aug 5 14:20:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:20:13.850095 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286826b7-af77-4289-b061-0192858be98c None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:20:13.851649 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-286826b7-af77-4289-b061-0192858be98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:20:13.954444 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286826b7-af77-4289-b061-0192858be98c None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:20:14.025826 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecdd45f-6c4b-417b-8e9e-a7b0cb35527f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:20:14.026432 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecdd45f-6c4b-417b-8e9e-a7b0cb35527f service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:20:14.026705 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecdd45f-6c4b-417b-8e9e-a7b0cb35527f service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:20:14.026971 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecdd45f-6c4b-417b-8e9e-a7b0cb35527f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:20:14.028624 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-8ecdd45f-6c4b-417b-8e9e-a7b0cb35527f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:20:14.037725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:20:14.138382 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecdd45f-6c4b-417b-8e9e-a7b0cb35527f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:20:14.139806 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecdd45f-6c4b-417b-8e9e-a7b0cb35527f service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:20:14.157507 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 760/1493] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:20:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:20:23.809101 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=82 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:20:24.155220 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=50 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:21:40.764436 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a90a4-0bcd-40d8-a218-141dbc8f58bf None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:21:40.765026 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a90a4-0bcd-40d8-a218-141dbc8f58bf None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:21:40.765026 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a90a4-0bcd-40d8-a218-141dbc8f58bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:21:40.840452 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-670a90a4-0bcd-40d8-a218-141dbc8f58bf None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:21:40.841595 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-670a90a4-0bcd-40d8-a218-141dbc8f58bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:21:40.938789 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 734/1494] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:21:40.947076 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb745e96-eb80-49dc-a156-d48046673881 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:21:40.947566 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb745e96-eb80-49dc-a156-d48046673881 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:21:40.947682 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb745e96-eb80-49dc-a156-d48046673881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:21:41.021507 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-cb745e96-eb80-49dc-a156-d48046673881 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:21:41.028259 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-cb745e96-eb80-49dc-a156-d48046673881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:21:41.117816 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 761/1495] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:21:41.122707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a90a4-0bcd-40d8-a218-141dbc8f58bf None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:21:41.123891 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-670a90a4-0bcd-40d8-a218-141dbc8f58bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:21:41.135787 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a90a4-0bcd-40d8-a218-141dbc8f58bf None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:21:41.194932 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:21:41.229894 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d93e0-7303-4372-9e17-34783a2165f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:21:41.230502 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d93e0-7303-4372-9e17-34783a2165f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:21:41.230777 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d93e0-7303-4372-9e17-34783a2165f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:21:41.231046 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d93e0-7303-4372-9e17-34783a2165f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:21:41.231729 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14d93e0-7303-4372-9e17-34783a2165f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:21:41.232026 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14d93e0-7303-4372-9e17-34783a2165f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:21:41.233083 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: WARNING py.warnings [None req-f14d93e0-7303-4372-9e17-34783a2165f7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 05 14:21:41.233083 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: warnings.warn(msg) Aug 05 14:21:41.233083 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: Aug 05 14:21:41.233966 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14d93e0-7303-4372-9e17-34783a2165f7 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:21:41.245494 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 735/1496] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:21:41.253695 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb745e96-eb80-49dc-a156-d48046673881 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:21:41.262705 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb745e96-eb80-49dc-a156-d48046673881 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:21:41.323080 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:21:41.356957 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a230c89-0ab5-4855-942c-b4a708e2354d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:21:41.357565 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a230c89-0ab5-4855-942c-b4a708e2354d admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:21:41.357836 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a230c89-0ab5-4855-942c-b4a708e2354d admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:21:41.358098 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a230c89-0ab5-4855-942c-b4a708e2354d admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:21:41.358760 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a230c89-0ab5-4855-942c-b4a708e2354d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:21:41.360485 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: WARNING py.warnings [None req-4a230c89-0ab5-4855-942c-b4a708e2354d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 14:21:41.360485 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: warnings.warn(msg) Aug 05 14:21:41.360485 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: Aug 05 14:21:41.361659 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a230c89-0ab5-4855-942c-b4a708e2354d admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:21:41.488392 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 762/1497] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:21:41 2022] POST /identity/v3/projects => generated 379 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:21:41.493338 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d93e0-7303-4372-9e17-34783a2165f7 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:21:41.494851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f14d93e0-7303-4372-9e17-34783a2165f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:21:41.528226 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d93e0-7303-4372-9e17-34783a2165f7 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:21:41.621097 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b0dcf2-ddd4-4815-b601-c2327dade0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:21:41.621705 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b0dcf2-ddd4-4815-b601-c2327dade0de admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:21:41.621976 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b0dcf2-ddd4-4815-b601-c2327dade0de admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:21:41.622240 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b0dcf2-ddd4-4815-b601-c2327dade0de admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:21:41.622892 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b0dcf2-ddd4-4815-b601-c2327dade0de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:21:41.623889 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: WARNING py.warnings [None req-49b0dcf2-ddd4-4815-b601-c2327dade0de admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 14:21:41.623889 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: warnings.warn(msg) Aug 05 14:21:41.623889 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: Aug 05 14:21:41.625134 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b0dcf2-ddd4-4815-b601-c2327dade0de admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:21:41.726496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 736/1498] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:21:41 2022] POST /identity/v3/users => generated 290 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:21:41.730961 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a230c89-0ab5-4855-942c-b4a708e2354d admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:21:41.732517 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-4a230c89-0ab5-4855-942c-b4a708e2354d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:21:41.794067 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a230c89-0ab5-4855-942c-b4a708e2354d admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:21:41.878542 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:21:41.888514 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989cd9f-29d1-4a6c-a601-b0f027cdb6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:21:41.892057 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989cd9f-29d1-4a6c-a601-b0f027cdb6ce admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:21:41.892057 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989cd9f-29d1-4a6c-a601-b0f027cdb6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:21:41.892057 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989cd9f-29d1-4a6c-a601-b0f027cdb6ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:21:41.892057 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7989cd9f-29d1-4a6c-a601-b0f027cdb6ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:21:41.892057 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: WARNING py.warnings [None req-7989cd9f-29d1-4a6c-a601-b0f027cdb6ce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 05 14:21:41.892057 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: warnings.warn(msg) Aug 05 14:21:41.892057 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: Aug 05 14:21:41.892676 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7989cd9f-29d1-4a6c-a601-b0f027cdb6ce admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:21:41.905482 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 763/1499] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:21:41.908686 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b0dcf2-ddd4-4815-b601-c2327dade0de admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:21:41.910538 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-49b0dcf2-ddd4-4815-b601-c2327dade0de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:21:41.922566 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b0dcf2-ddd4-4815-b601-c2327dade0de admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:21:41.947371 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:21:42.017193 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c7eda4-377c-4b16-aa1c-f0103791dec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:21:42.017811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c7eda4-377c-4b16-aa1c-f0103791dec0 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:21:42.018090 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c7eda4-377c-4b16-aa1c-f0103791dec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:21:42.018365 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c7eda4-377c-4b16-aa1c-f0103791dec0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:21:42.019057 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c7eda4-377c-4b16-aa1c-f0103791dec0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:21:42.020068 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: WARNING py.warnings [None req-46c7eda4-377c-4b16-aa1c-f0103791dec0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 05 14:21:42.020068 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: warnings.warn(msg) Aug 05 14:21:42.020068 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: Aug 05 14:21:42.020987 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c7eda4-377c-4b16-aa1c-f0103791dec0 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:21:42.033689 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 737/1500] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:21:42.038035 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989cd9f-29d1-4a6c-a601-b0f027cdb6ce admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:21:42.040197 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-7989cd9f-29d1-4a6c-a601-b0f027cdb6ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:21:42.073925 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989cd9f-29d1-4a6c-a601-b0f027cdb6ce admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:21:42.168056 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08edbc74-5ec5-498e-8caa-7d45d41e2170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:21:42.168765 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08edbc74-5ec5-498e-8caa-7d45d41e2170 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:21:42.169030 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08edbc74-5ec5-498e-8caa-7d45d41e2170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:21:42.169303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08edbc74-5ec5-498e-8caa-7d45d41e2170 admin admin] PATH_INFO: `/v3/projects/cba4de4ea7384a75bd9c6a76359dfbd7/users/23586733d8a443cabda7ccbe4fbf7243/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:21:42.191294 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08edbc74-5ec5-498e-8caa-7d45d41e2170 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cba4de4ea7384a75bd9c6a76359dfbd7, user_id=23586733d8a443cabda7ccbe4fbf7243, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:21:42.192313 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: WARNING py.warnings [None req-08edbc74-5ec5-498e-8caa-7d45d41e2170 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 14:21:42.192313 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: warnings.warn(msg) Aug 05 14:21:42.192313 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: Aug 05 14:21:42.194280 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08edbc74-5ec5-498e-8caa-7d45d41e2170 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:21:42.286981 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 764/1501] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:21:42 2022] PUT /identity/v3/projects/cba4de4ea7384a75bd9c6a76359dfbd7/users/23586733d8a443cabda7ccbe4fbf7243/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:21:42.300099 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c7f98-1476-4111-aa51-26cf545e91e9 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:21:42.300386 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c7f98-1476-4111-aa51-26cf545e91e9 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:21:42.300690 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c7f98-1476-4111-aa51-26cf545e91e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:21:42.357666 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-e45c7f98-1476-4111-aa51-26cf545e91e9 None None] MFA Rules not processed for user `23586733d8a443cabda7ccbe4fbf7243`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:21:42.358809 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e45c7f98-1476-4111-aa51-26cf545e91e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:21:42.434841 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 738/1502] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:21:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:21:42.448056 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b962877-3f6b-4e14-a7b2-23164be665f2 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:21:42.448342 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b962877-3f6b-4e14-a7b2-23164be665f2 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:21:42.448648 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b962877-3f6b-4e14-a7b2-23164be665f2 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:21:42.449981 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 765/1503] 10.222.0.26 () {54 vars in 964 bytes} [Fri Aug 5 14:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:21:42.454662 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e06272-045e-44f7-b941-a00c4d692cab None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:21:42.454957 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e06272-045e-44f7-b941-a00c4d692cab None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:21:42.455230 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e06272-045e-44f7-b941-a00c4d692cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:21:42.487950 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:21:42.532658 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-e3e06272-045e-44f7-b941-a00c4d692cab None None] MFA Rules not processed for user `88d6a1bae9834e2e91d4ab3a4c00959b`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:21:42.533989 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e3e06272-045e-44f7-b941-a00c4d692cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:21:42.608927 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 739/1504] 10.222.0.26 () {58 vars in 1061 bytes} [Fri Aug 5 14:21:42 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:21:42.616677 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b962877-3f6b-4e14-a7b2-23164be665f2 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:21:42.618225 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-7b962877-3f6b-4e14-a7b2-23164be665f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:21:42.667898 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:21:42.722171 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b962877-3f6b-4e14-a7b2-23164be665f2 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:21:42.794019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74a7777-b9db-4caf-83a1-b02ca5d9bfdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:21:42.794630 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a7777-b9db-4caf-83a1-b02ca5d9bfdc service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:21:42.794907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a7777-b9db-4caf-83a1-b02ca5d9bfdc service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:21:42.795177 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a7777-b9db-4caf-83a1-b02ca5d9bfdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:21:42.796824 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-e74a7777-b9db-4caf-83a1-b02ca5d9bfdc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:21:42.893300 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74a7777-b9db-4caf-83a1-b02ca5d9bfdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:21:42.894654 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74a7777-b9db-4caf-83a1-b02ca5d9bfdc service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:21:42.898239 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:21:42.912879 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 766/1505] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:21:42 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:21:42.957755 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29b532-ed03-47a0-b075-988e9dd7cdd9 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:21:42.958191 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29b532-ed03-47a0-b075-988e9dd7cdd9 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:21:42.958302 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29b532-ed03-47a0-b075-988e9dd7cdd9 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:21:42.959549 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 740/1506] 10.222.0.26 () {52 vars in 830 bytes} [Fri Aug 5 14:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:21:42.963112 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74a7777-b9db-4caf-83a1-b02ca5d9bfdc service cinder] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:21:42.964311 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-e74a7777-b9db-4caf-83a1-b02ca5d9bfdc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:21:43.064224 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74a7777-b9db-4caf-83a1-b02ca5d9bfdc service cinder] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:21:43.156628 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e544f-a410-42e9-aa27-79d538f32dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:21:43.157197 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e544f-a410-42e9-aa27-79d538f32dc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:21:43.157480 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e544f-a410-42e9-aa27-79d538f32dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:21:43.157756 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e544f-a410-42e9-aa27-79d538f32dc9 admin admin] PATH_INFO: `/v3/users/23586733d8a443cabda7ccbe4fbf7243` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:21:43.170443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e544f-a410-42e9-aa27-79d538f32dc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23586733d8a443cabda7ccbe4fbf7243)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:21:43.171454 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: WARNING py.warnings [None req-9a5e544f-a410-42e9-aa27-79d538f32dc9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 14:21:43.171454 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: warnings.warn(msg) Aug 05 14:21:43.171454 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: Aug 05 14:21:43.172702 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e544f-a410-42e9-aa27-79d538f32dc9 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:21:43.182190 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:21:43.366712 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9a5e544f-a410-42e9-aa27-79d538f32dc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23586733d8a443cabda7ccbe4fbf7243'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:21:43.380632 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9a5e544f-a410-42e9-aa27-79d538f32dc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23586733d8a443cabda7ccbe4fbf7243'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:21:43.381199 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9a5e544f-a410-42e9-aa27-79d538f32dc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23586733d8a443cabda7ccbe4fbf7243'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:21:43.381615 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9a5e544f-a410-42e9-aa27-79d538f32dc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23586733d8a443cabda7ccbe4fbf7243'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:21:43.405629 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9a5e544f-a410-42e9-aa27-79d538f32dc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23586733d8a443cabda7ccbe4fbf7243'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:21:43.483696 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 767/1507] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:21:42 2022] DELETE /identity/v3/users/23586733d8a443cabda7ccbe4fbf7243 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:21:43.487972 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff62e75-97a8-4d42-a232-1aec4fe2f24f None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:21:43.488256 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff62e75-97a8-4d42-a232-1aec4fe2f24f None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:21:43.488564 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff62e75-97a8-4d42-a232-1aec4fe2f24f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:21:43.547125 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-7ff62e75-97a8-4d42-a232-1aec4fe2f24f None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:21:43.548330 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-7ff62e75-97a8-4d42-a232-1aec4fe2f24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:21:43.621568 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:21:43.634695 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 741/1508] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:21:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:21:43.645135 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e544f-a410-42e9-aa27-79d538f32dc9 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:21:43.646336 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-9a5e544f-a410-42e9-aa27-79d538f32dc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:21:43.745003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e544f-a410-42e9-aa27-79d538f32dc9 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:21:43.797657 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:21:43.817028 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb256a-1ecd-432f-9a26-f4a31b1addc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:21:43.817644 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb256a-1ecd-432f-9a26-f4a31b1addc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:21:43.817920 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb256a-1ecd-432f-9a26-f4a31b1addc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:21:43.818188 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb256a-1ecd-432f-9a26-f4a31b1addc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:21:43.819811 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-0dbb256a-1ecd-432f-9a26-f4a31b1addc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:21:43.931241 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbb256a-1ecd-432f-9a26-f4a31b1addc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:21:43.932802 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbb256a-1ecd-432f-9a26-f4a31b1addc7 service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:21:43.951893 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 768/1509] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:21:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:21:44.396894 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff62e75-97a8-4d42-a232-1aec4fe2f24f None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:21:44.398119 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-7ff62e75-97a8-4d42-a232-1aec4fe2f24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:21:44.410253 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff62e75-97a8-4d42-a232-1aec4fe2f24f None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:21:44.510030 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5448b1cd-f42b-4560-82e0-5bb1561f208f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:21:44.510664 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448b1cd-f42b-4560-82e0-5bb1561f208f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:21:44.510935 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448b1cd-f42b-4560-82e0-5bb1561f208f admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:21:44.511198 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448b1cd-f42b-4560-82e0-5bb1561f208f admin admin] PATH_INFO: `/v3/projects/cba4de4ea7384a75bd9c6a76359dfbd7` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:21:44.523543 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5448b1cd-f42b-4560-82e0-5bb1561f208f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cba4de4ea7384a75bd9c6a76359dfbd7)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:21:44.525138 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: WARNING py.warnings [None req-5448b1cd-f42b-4560-82e0-5bb1561f208f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 14:21:44.525138 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: warnings.warn(msg) Aug 05 14:21:44.525138 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: Aug 05 14:21:44.526339 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5448b1cd-f42b-4560-82e0-5bb1561f208f admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:21:44.645488 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-5448b1cd-f42b-4560-82e0-5bb1561f208f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cba4de4ea7384a75bd9c6a76359dfbd7', 'request_id': 'req-5448b1cd-f42b-4560-82e0-5bb1561f208f'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:21:44.658963 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-5448b1cd-f42b-4560-82e0-5bb1561f208f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cba4de4ea7384a75bd9c6a76359dfbd7', 'request_id': 'req-5448b1cd-f42b-4560-82e0-5bb1561f208f'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:21:44.746655 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-5448b1cd-f42b-4560-82e0-5bb1561f208f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cba4de4ea7384a75bd9c6a76359dfbd7', 'request_id': 'req-5448b1cd-f42b-4560-82e0-5bb1561f208f'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:21:44.769123 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-5448b1cd-f42b-4560-82e0-5bb1561f208f admin admin] The token cache is being invalidate because project cba4de4ea7384a75bd9c6a76359dfbd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:21:44.769409 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-5448b1cd-f42b-4560-82e0-5bb1561f208f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:21:44.769831 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-5448b1cd-f42b-4560-82e0-5bb1561f208f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:21:44.771248 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 742/1510] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:21:44 2022] DELETE /identity/v3/projects/cba4de4ea7384a75bd9c6a76359dfbd7 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:21:53.949638 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=62 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:21:54.767205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:23:12.488560 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1753c-2d2f-4527-929a-6d52893e0385 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:23:12.489188 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1753c-2d2f-4527-929a-6d52893e0385 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:23:12.489188 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1753c-2d2f-4527-929a-6d52893e0385 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:23:12.490480 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 769/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 14:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:23:12.495508 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cdf87-9424-4e85-9162-ddbdd027884a None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:23:12.495937 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cdf87-9424-4e85-9162-ddbdd027884a None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:23:12.496041 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cdf87-9424-4e85-9162-ddbdd027884a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:23:12.576636 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-c17cdf87-9424-4e85-9162-ddbdd027884a None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:23:12.577839 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c17cdf87-9424-4e85-9162-ddbdd027884a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:23:12.679183 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 743/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 14:23:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:23:12.831875 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1753c-2d2f-4527-929a-6d52893e0385 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:23:12.834058 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-b9a1753c-2d2f-4527-929a-6d52893e0385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:23:12.952057 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1753c-2d2f-4527-929a-6d52893e0385 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:23:12.957664 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee1219-f9a6-4f68-b10a-a4493266a675 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:23:12.958279 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee1219-f9a6-4f68-b10a-a4493266a675 service nova] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:23:12.958554 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee1219-f9a6-4f68-b10a-a4493266a675 service nova] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:23:12.958824 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee1219-f9a6-4f68-b10a-a4493266a675 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:23:12.967533 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee1219-f9a6-4f68-b10a-a4493266a675 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:23:12.969150 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee1219-f9a6-4f68-b10a-a4493266a675 service nova] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:23:12.979192 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 770/1513] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:23:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:23:13.814008 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e291f8a-9467-44d5-a7a0-1e1275882d97 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:23:13.814538 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e291f8a-9467-44d5-a7a0-1e1275882d97 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:23:13.814538 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e291f8a-9467-44d5-a7a0-1e1275882d97 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:23:13.815846 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 744/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 14:23:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:23:13.821365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c8412-a370-4830-869f-46e03c9d9c7a None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:23:13.821793 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c8412-a370-4830-869f-46e03c9d9c7a None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:23:13.821893 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c8412-a370-4830-869f-46e03c9d9c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:23:13.841070 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-843c8412-a370-4830-869f-46e03c9d9c7a None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:23:13.842268 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-843c8412-a370-4830-869f-46e03c9d9c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:23:13.850622 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 771/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 14:23:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:23:14.003151 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e291f8a-9467-44d5-a7a0-1e1275882d97 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:23:14.018112 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e291f8a-9467-44d5-a7a0-1e1275882d97 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:23:14.024184 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e624c-c707-4870-9b8c-5c825bfca1ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:23:14.024831 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e624c-c707-4870-9b8c-5c825bfca1ad service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:23:14.025111 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e624c-c707-4870-9b8c-5c825bfca1ad service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:23:14.025387 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e624c-c707-4870-9b8c-5c825bfca1ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:23:14.033422 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839e624c-c707-4870-9b8c-5c825bfca1ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:23:14.035008 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839e624c-c707-4870-9b8c-5c825bfca1ad service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:23:14.044487 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 745/1516] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:23:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:23:14.481398 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc85a91-b509-4850-bb77-018cfb25d0b3 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:23:14.481683 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc85a91-b509-4850-bb77-018cfb25d0b3 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:23:14.481955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc85a91-b509-4850-bb77-018cfb25d0b3 None None] PATH_INFO: `/v3` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:23:14.483241 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 772/1517] 10.222.0.13 () {54 vars in 925 bytes} [Fri Aug 5 14:23:14 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:23:14.490154 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5836bc1-7404-4fe8-bc5c-2e887ec201e2 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:23:14.490440 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5836bc1-7404-4fe8-bc5c-2e887ec201e2 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:23:14.490718 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5836bc1-7404-4fe8-bc5c-2e887ec201e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:23:14.551769 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-f5836bc1-7404-4fe8-bc5c-2e887ec201e2 None None] MFA Rules not processed for user `fff761f6f81a46098f94bd2ec2be9c1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:23:14.553002 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f5836bc1-7404-4fe8-bc5c-2e887ec201e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:23:14.569301 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:23:14.624035 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 746/1518] 10.222.0.13 () {58 vars in 1015 bytes} [Fri Aug 5 14:23:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:23:23.762275 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d9949-aa1a-49c3-801e-7b673971d51d None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:23:23.762802 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d9949-aa1a-49c3-801e-7b673971d51d None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:23:23.762802 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d9949-aa1a-49c3-801e-7b673971d51d None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:23:23.764060 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 773/1519] 10.222.0.26 () {54 vars in 961 bytes} [Fri Aug 5 14:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:23:23.769099 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db92a2-f0dc-439f-93f8-62e0fd6c0fbc None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:23:23.769504 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db92a2-f0dc-439f-93f8-62e0fd6c0fbc None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:23:23.769642 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db92a2-f0dc-439f-93f8-62e0fd6c0fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:23:23.790667 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-18db92a2-f0dc-439f-93f8-62e0fd6c0fbc None None] MFA Rules not processed for user `fff761f6f81a46098f94bd2ec2be9c1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:23:23.791846 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-18db92a2-f0dc-439f-93f8-62e0fd6c0fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:23:23.800184 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 747/1520] 10.222.0.26 () {58 vars in 1058 bytes} [Fri Aug 5 14:23:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:23:23.807548 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d9949-aa1a-49c3-801e-7b673971d51d None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:23:23.821664 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d9949-aa1a-49c3-801e-7b673971d51d None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:23:23.826668 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b314b8-4f2e-48eb-b443-11105210835e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff761f6f81a46098f94bd2ec2be9c1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:23:23.827269 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b314b8-4f2e-48eb-b443-11105210835e service placement] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:23:23.827570 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b314b8-4f2e-48eb-b443-11105210835e service placement] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:23:23.827869 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b314b8-4f2e-48eb-b443-11105210835e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:23:23.837166 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b314b8-4f2e-48eb-b443-11105210835e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:23:23.838771 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b314b8-4f2e-48eb-b443-11105210835e service placement] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:23:23.848428 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 774/1521] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:23:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:23:25.275182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4df7f-bbe9-4036-a09e-b9098c2b7960 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:23:25.275728 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4df7f-bbe9-4036-a09e-b9098c2b7960 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:23:25.275728 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4df7f-bbe9-4036-a09e-b9098c2b7960 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:23:25.277091 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 748/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 14:23:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:23:25.281852 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4739186-568d-4d70-8fb0-5ed286c3ad2f None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:23:25.282284 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4739186-568d-4d70-8fb0-5ed286c3ad2f None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:23:25.282391 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4739186-568d-4d70-8fb0-5ed286c3ad2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:23:25.301804 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-b4739186-568d-4d70-8fb0-5ed286c3ad2f None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:23:25.302997 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-b4739186-568d-4d70-8fb0-5ed286c3ad2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:23:25.311844 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 775/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 14:23:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:23:25.466652 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b4df7f-bbe9-4036-a09e-b9098c2b7960 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:23:25.483758 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b4df7f-bbe9-4036-a09e-b9098c2b7960 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:23:25.490687 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66225f35-6561-4230-9587-a00f29d3f6d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:23:25.491284 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66225f35-6561-4230-9587-a00f29d3f6d7 service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:23:25.492191 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66225f35-6561-4230-9587-a00f29d3f6d7 service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:23:25.492634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66225f35-6561-4230-9587-a00f29d3f6d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:23:25.500626 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66225f35-6561-4230-9587-a00f29d3f6d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:23:25.502182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66225f35-6561-4230-9587-a00f29d3f6d7 service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:23:25.512006 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 749/1524] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:23:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:23:26.334300 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ed83c-1905-476d-8782-d86864359dd6 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:23:26.334881 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ed83c-1905-476d-8782-d86864359dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:23:26.334881 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ed83c-1905-476d-8782-d86864359dd6 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:23:26.336891 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 776/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 14:23:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:23:26.341784 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6068489-8caf-4724-91fc-291158b6c07d None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:23:26.342230 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6068489-8caf-4724-91fc-291158b6c07d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:23:26.342329 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6068489-8caf-4724-91fc-291158b6c07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:23:26.361347 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-e6068489-8caf-4724-91fc-291158b6c07d None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:23:26.362541 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e6068489-8caf-4724-91fc-291158b6c07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:23:26.371034 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 750/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 14:23:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:23:26.521573 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ed83c-1905-476d-8782-d86864359dd6 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:23:26.536865 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ed83c-1905-476d-8782-d86864359dd6 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:23:26.543204 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a5859-df1d-4c2f-aef6-21956d63af94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:23:26.543801 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a5859-df1d-4c2f-aef6-21956d63af94 service nova] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:23:26.544073 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a5859-df1d-4c2f-aef6-21956d63af94 service nova] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:23:26.544341 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a5859-df1d-4c2f-aef6-21956d63af94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:23:26.552199 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7a5859-df1d-4c2f-aef6-21956d63af94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:23:26.553814 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7a5859-df1d-4c2f-aef6-21956d63af94 service nova] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:23:26.563610 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 777/1527] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:23:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:23:30.343211 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6c08a-be31-44d7-95ad-4b867539f073 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:23:30.343754 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6c08a-be31-44d7-95ad-4b867539f073 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:23:30.343754 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6c08a-be31-44d7-95ad-4b867539f073 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:23:30.345196 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 751/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 14:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:23:30.351459 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de050c0e-141d-4709-a23c-95858ca2cf72 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:23:30.352126 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de050c0e-141d-4709-a23c-95858ca2cf72 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:23:30.352126 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de050c0e-141d-4709-a23c-95858ca2cf72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:23:30.372829 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-de050c0e-141d-4709-a23c-95858ca2cf72 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:23:30.374033 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-de050c0e-141d-4709-a23c-95858ca2cf72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:23:30.381960 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 778/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 14:23:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:23:30.455350 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6c08a-be31-44d7-95ad-4b867539f073 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:23:30.457043 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-11a6c08a-be31-44d7-95ad-4b867539f073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:23:30.536583 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6c08a-be31-44d7-95ad-4b867539f073 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:23:30.540245 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a891e4-d2c6-43ff-b0b2-1fae14d42639 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:23:30.540884 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a891e4-d2c6-43ff-b0b2-1fae14d42639 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:23:30.541167 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a891e4-d2c6-43ff-b0b2-1fae14d42639 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:23:30.541434 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a891e4-d2c6-43ff-b0b2-1fae14d42639 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:23:30.550156 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a891e4-d2c6-43ff-b0b2-1fae14d42639 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:23:30.551690 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a891e4-d2c6-43ff-b0b2-1fae14d42639 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:23:30.561417 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 752/1530] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:23:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:23:32.577289 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edd87f-6def-4d22-a9fc-ce4caddf4733 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:23:32.577818 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edd87f-6def-4d22-a9fc-ce4caddf4733 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:23:32.577937 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edd87f-6def-4d22-a9fc-ce4caddf4733 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:23:32.579220 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 779/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 14:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:23:32.584222 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953761e4-e674-44b9-ab33-6215a5ed2b59 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:23:32.584719 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953761e4-e674-44b9-ab33-6215a5ed2b59 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:23:32.584825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953761e4-e674-44b9-ab33-6215a5ed2b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:23:32.604210 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-953761e4-e674-44b9-ab33-6215a5ed2b59 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:23:32.605440 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-953761e4-e674-44b9-ab33-6215a5ed2b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:23:32.613522 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 753/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 14:23:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:23:32.649871 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3edd87f-6def-4d22-a9fc-ce4caddf4733 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:23:32.660449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3edd87f-6def-4d22-a9fc-ce4caddf4733 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:23:32.664156 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe2c5e-9e6d-4117-a7bf-471b131fa9c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:23:32.664787 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe2c5e-9e6d-4117-a7bf-471b131fa9c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:23:32.665061 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe2c5e-9e6d-4117-a7bf-471b131fa9c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:23:32.665326 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe2c5e-9e6d-4117-a7bf-471b131fa9c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:23:32.673918 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbe2c5e-9e6d-4117-a7bf-471b131fa9c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:23:32.675323 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbe2c5e-9e6d-4117-a7bf-471b131fa9c8 service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:23:32.685428 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 780/1533] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:23:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 14:23:36.449161 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4311ce-a05f-49b4-b54a-84b1d9305364 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:23:36.449739 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4311ce-a05f-49b4-b54a-84b1d9305364 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:23:36.449739 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4311ce-a05f-49b4-b54a-84b1d9305364 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:23:36.451028 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 754/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 14:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:23:36.456104 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb44d662-6fb9-45e7-a9b7-c4440f4bfce1 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:23:36.456511 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb44d662-6fb9-45e7-a9b7-c4440f4bfce1 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:23:36.456681 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb44d662-6fb9-45e7-a9b7-c4440f4bfce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:23:36.464939 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=81 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:23:36.477763 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-fb44d662-6fb9-45e7-a9b7-c4440f4bfce1 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:23:36.478987 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-fb44d662-6fb9-45e7-a9b7-c4440f4bfce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:23:36.491395 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 781/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 14:23:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:23:36.795662 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4311ce-a05f-49b4-b54a-84b1d9305364 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:23:36.797374 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-db4311ce-a05f-49b4-b54a-84b1d9305364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:23:36.862325 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:23:36.875160 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4311ce-a05f-49b4-b54a-84b1d9305364 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:23:36.878772 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6505cd43-b5d5-4bc7-858d-fc36422c52f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:23:36.879387 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505cd43-b5d5-4bc7-858d-fc36422c52f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:23:36.879667 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505cd43-b5d5-4bc7-858d-fc36422c52f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:23:36.879939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505cd43-b5d5-4bc7-858d-fc36422c52f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:23:36.888886 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6505cd43-b5d5-4bc7-858d-fc36422c52f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:23:36.890592 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6505cd43-b5d5-4bc7-858d-fc36422c52f5 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:23:36.900783 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 755/1536] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:23:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:23:38.917905 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60bc565-3b65-43f1-a4f6-ff04e5ec05a4 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:23:38.918446 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60bc565-3b65-43f1-a4f6-ff04e5ec05a4 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:23:38.918446 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60bc565-3b65-43f1-a4f6-ff04e5ec05a4 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:23:38.919878 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 782/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 14:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:23:38.925199 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdc274-0bae-438b-8069-cf928ee263b7 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:23:38.925627 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdc274-0bae-438b-8069-cf928ee263b7 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:23:38.925755 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdc274-0bae-438b-8069-cf928ee263b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:23:38.945318 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-a2fdc274-0bae-438b-8069-cf928ee263b7 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:23:38.946545 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a2fdc274-0bae-438b-8069-cf928ee263b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:23:38.954407 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 756/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 14:23:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:23:39.256272 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60bc565-3b65-43f1-a4f6-ff04e5ec05a4 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:23:39.259005 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-f60bc565-3b65-43f1-a4f6-ff04e5ec05a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:23:39.272141 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60bc565-3b65-43f1-a4f6-ff04e5ec05a4 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:23:39.275656 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b837f-9737-430c-9c7c-ae6a14a4e630 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:23:39.276265 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b837f-9737-430c-9c7c-ae6a14a4e630 service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:23:39.276590 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b837f-9737-430c-9c7c-ae6a14a4e630 service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:23:39.276867 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b837f-9737-430c-9c7c-ae6a14a4e630 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:23:39.285620 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9b837f-9737-430c-9c7c-ae6a14a4e630 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:23:39.287045 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9b837f-9737-430c-9c7c-ae6a14a4e630 service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:23:39.296356 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 783/1539] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:23:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:23:48.940235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=11 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:23:49.292149 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=8 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:03.751541 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a15a1-60d8-4c16-a8f7-f2ebb0742444 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:03.752204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a15a1-60d8-4c16-a8f7-f2ebb0742444 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:03.752204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a15a1-60d8-4c16-a8f7-f2ebb0742444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:03.773424 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-382a15a1-60d8-4c16-a8f7-f2ebb0742444 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:03.775028 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-382a15a1-60d8-4c16-a8f7-f2ebb0742444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:03.788555 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 757/1540] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:03.797183 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a826d52-27a6-4e03-839b-76ceff8b61f0 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:03.797735 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a826d52-27a6-4e03-839b-76ceff8b61f0 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:03.797735 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a826d52-27a6-4e03-839b-76ceff8b61f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:03.824779 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-7a826d52-27a6-4e03-839b-76ceff8b61f0 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:03.826108 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-7a826d52-27a6-4e03-839b-76ceff8b61f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:03.841972 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 784/1541] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:03.846654 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a15a1-60d8-4c16-a8f7-f2ebb0742444 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:03.860346 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a15a1-60d8-4c16-a8f7-f2ebb0742444 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:03.868899 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906e393-0727-473a-8c63-7f5a5adbcb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:03.870065 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906e393-0727-473a-8c63-7f5a5adbcb87 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:03.870599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906e393-0727-473a-8c63-7f5a5adbcb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:03.871185 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906e393-0727-473a-8c63-7f5a5adbcb87 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:03.872155 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f906e393-0727-473a-8c63-7f5a5adbcb87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:25:03.872730 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f906e393-0727-473a-8c63-7f5a5adbcb87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:03.874791 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f906e393-0727-473a-8c63-7f5a5adbcb87 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:03.893523 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 758/1542] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:25:03.903000 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a826d52-27a6-4e03-839b-76ceff8b61f0 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:03.918030 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a826d52-27a6-4e03-839b-76ceff8b61f0 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:03.929233 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ce87bd-3cf0-46d5-a54c-4d15510d6abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:03.929875 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce87bd-3cf0-46d5-a54c-4d15510d6abf admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:03.930150 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce87bd-3cf0-46d5-a54c-4d15510d6abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:03.930421 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce87bd-3cf0-46d5-a54c-4d15510d6abf admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:03.931094 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ce87bd-3cf0-46d5-a54c-4d15510d6abf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:03.933005 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ce87bd-3cf0-46d5-a54c-4d15510d6abf admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:04.098845 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 785/1543] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:25:03 2022] POST /identity/v3/projects => generated 391 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:04.103294 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906e393-0727-473a-8c63-7f5a5adbcb87 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:04.113961 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906e393-0727-473a-8c63-7f5a5adbcb87 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:04.201757 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa56ead-b9ad-45be-b7fe-291f8477e984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:04.202408 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa56ead-b9ad-45be-b7fe-291f8477e984 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:04.202685 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa56ead-b9ad-45be-b7fe-291f8477e984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:04.202958 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa56ead-b9ad-45be-b7fe-291f8477e984 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:04.203640 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa56ead-b9ad-45be-b7fe-291f8477e984 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:04.205648 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa56ead-b9ad-45be-b7fe-291f8477e984 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:04.243511 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca21c7-d2b7-4eda-bd29-5c6a29d05f47 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:04.243828 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca21c7-d2b7-4eda-bd29-5c6a29d05f47 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:04.244202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca21c7-d2b7-4eda-bd29-5c6a29d05f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:04.273019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-97ca21c7-d2b7-4eda-bd29-5c6a29d05f47 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:04.274347 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-97ca21c7-d2b7-4eda-bd29-5c6a29d05f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:04.289206 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 786/1544] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:04.298328 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee8634-c7fa-43bc-a034-3d50915b8146 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:04.298647 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee8634-c7fa-43bc-a034-3d50915b8146 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:04.298882 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee8634-c7fa-43bc-a034-3d50915b8146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:04.328427 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-21ee8634-c7fa-43bc-a034-3d50915b8146 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:04.329523 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-21ee8634-c7fa-43bc-a034-3d50915b8146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:04.339287 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 759/1545] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:25:04 2022] POST /identity/v3/users => generated 296 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:04.343158 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa56ead-b9ad-45be-b7fe-291f8477e984 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:04.343319 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 787/1546] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:25:04.348355 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee8634-c7fa-43bc-a034-3d50915b8146 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:04.359375 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa56ead-b9ad-45be-b7fe-291f8477e984 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:04.361542 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee8634-c7fa-43bc-a034-3d50915b8146 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:04.368570 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e4718-b75a-47b2-adb3-cea447e26830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:04.369202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e4718-b75a-47b2-adb3-cea447e26830 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:04.369474 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e4718-b75a-47b2-adb3-cea447e26830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:04.369746 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e4718-b75a-47b2-adb3-cea447e26830 admin admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:04.370546 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e4718-b75a-47b2-adb3-cea447e26830 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:25:04.370843 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e4718-b75a-47b2-adb3-cea447e26830 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:04.371851 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: WARNING py.warnings [None req-593e4718-b75a-47b2-adb3-cea447e26830 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 05 14:25:04.371851 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: warnings.warn(msg) Aug 05 14:25:04.371851 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: Aug 05 14:25:04.372169 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab30926-0972-416c-8344-a1c999242589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:04.372780 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab30926-0972-416c-8344-a1c999242589 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:04.375784 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e4718-b75a-47b2-adb3-cea447e26830 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:04.375907 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab30926-0972-416c-8344-a1c999242589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:04.375907 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab30926-0972-416c-8344-a1c999242589 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:04.375907 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab30926-0972-416c-8344-a1c999242589 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:04.375907 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab30926-0972-416c-8344-a1c999242589 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:04.394405 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 760/1548] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:04.394631 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 788/1548] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:25:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:25:04.396065 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab30926-0972-416c-8344-a1c999242589 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:04.406837 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e4718-b75a-47b2-adb3-cea447e26830 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:04.411636 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab30926-0972-416c-8344-a1c999242589 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:04.422375 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e4718-b75a-47b2-adb3-cea447e26830 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:04.423390 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b0526b-9728-48a2-ad4c-f69b4a8ac378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:04.429040 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0526b-9728-48a2-ad4c-f69b4a8ac378 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:04.429536 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0526b-9728-48a2-ad4c-f69b4a8ac378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:04.429867 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0526b-9728-48a2-ad4c-f69b4a8ac378 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:04.430628 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b0526b-9728-48a2-ad4c-f69b4a8ac378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:04.432669 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b0526b-9728-48a2-ad4c-f69b4a8ac378 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:04.445429 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 761/1549] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:04.446427 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d7fd91-4594-4cfa-b089-0d59fc2e8511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:04.447082 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7fd91-4594-4cfa-b089-0d59fc2e8511 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:04.447411 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7fd91-4594-4cfa-b089-0d59fc2e8511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:04.447683 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7fd91-4594-4cfa-b089-0d59fc2e8511 admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:04.448464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d7fd91-4594-4cfa-b089-0d59fc2e8511 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:04.450050 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b0526b-9728-48a2-ad4c-f69b4a8ac378 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:04.450632 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d7fd91-4594-4cfa-b089-0d59fc2e8511 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:04.467548 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b0526b-9728-48a2-ad4c-f69b4a8ac378 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:04.480214 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85d8ff-abd2-4b91-ab3c-e762c2c6d5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:04.480948 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85d8ff-abd2-4b91-ab3c-e762c2c6d5b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:04.481225 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85d8ff-abd2-4b91-ab3c-e762c2c6d5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:04.481497 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85d8ff-abd2-4b91-ab3c-e762c2c6d5b5 admin admin] PATH_INFO: `/v3/projects/8725df4ddc2b4ec8accc5b0bd7b39a49/users/b9d62fe4e6de4308a3a51be3e900625f/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:04.510445 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e85d8ff-abd2-4b91-ab3c-e762c2c6d5b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8725df4ddc2b4ec8accc5b0bd7b39a49, user_id=b9d62fe4e6de4308a3a51be3e900625f, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:04.511835 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: WARNING py.warnings [None req-4e85d8ff-abd2-4b91-ab3c-e762c2c6d5b5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 14:25:04.511835 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: warnings.warn(msg) Aug 05 14:25:04.511835 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: Aug 05 14:25:04.513841 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e85d8ff-abd2-4b91-ab3c-e762c2c6d5b5 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:04.570787 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 789/1550] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:25:04 2022] POST /identity/v3/projects => generated 369 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:25:04.572257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22cb1d9-44ae-46c3-a021-86596906e6bb None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:04.572619 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22cb1d9-44ae-46c3-a021-86596906e6bb None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:04.572893 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22cb1d9-44ae-46c3-a021-86596906e6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:04.606340 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-e22cb1d9-44ae-46c3-a021-86596906e6bb None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:04.611613 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-e22cb1d9-44ae-46c3-a021-86596906e6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:04.655365 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 762/1551] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:25:04 2022] PUT /identity/v3/projects/8725df4ddc2b4ec8accc5b0bd7b39a49/users/b9d62fe4e6de4308a3a51be3e900625f/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:25:04.656289 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85d8ff-abd2-4b91-ab3c-e762c2c6d5b5 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:04.719571 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 790/1552] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:04.722156 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b3ee1-e855-4c7b-b8de-116a827b0163 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:04.722689 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b3ee1-e855-4c7b-b8de-116a827b0163 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:04.723202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b3ee1-e855-4c7b-b8de-116a827b0163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:04.749971 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-e91b3ee1-e855-4c7b-b8de-116a827b0163 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:04.751338 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-e91b3ee1-e855-4c7b-b8de-116a827b0163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:04.816007 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85d8ff-abd2-4b91-ab3c-e762c2c6d5b5 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:04.824990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3403d2-595c-4a29-8146-10d73edf1bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:04.825805 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3403d2-595c-4a29-8146-10d73edf1bf9 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:04.826227 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3403d2-595c-4a29-8146-10d73edf1bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:04.826649 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3403d2-595c-4a29-8146-10d73edf1bf9 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:04.827449 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3403d2-595c-4a29-8146-10d73edf1bf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:04.830189 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3403d2-595c-4a29-8146-10d73edf1bf9 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:04.864277 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 791/1553] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:04.865923 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b7637-7852-4661-b313-54437b4fe5ff None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:04.867115 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b7637-7852-4661-b313-54437b4fe5ff None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:04.867439 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b7637-7852-4661-b313-54437b4fe5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:04.898706 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-994b7637-7852-4661-b313-54437b4fe5ff None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:04.900754 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-994b7637-7852-4661-b313-54437b4fe5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:04.914995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 792/1554] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:04.914995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46821727-5072-4e0d-b1a3-16a2e7d789cb None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:04.914995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46821727-5072-4e0d-b1a3-16a2e7d789cb None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:04.914995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46821727-5072-4e0d-b1a3-16a2e7d789cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:04.930434 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 763/1555] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:25:04 2022] POST /identity/v3/users => generated 285 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:04.935133 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412fef32-a4ef-4ba4-bc27-d94d56dd91d6 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:04.935133 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412fef32-a4ef-4ba4-bc27-d94d56dd91d6 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:04.935133 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412fef32-a4ef-4ba4-bc27-d94d56dd91d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:04.949198 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-46821727-5072-4e0d-b1a3-16a2e7d789cb None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:04.952050 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-46821727-5072-4e0d-b1a3-16a2e7d789cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:04.956853 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-412fef32-a4ef-4ba4-bc27-d94d56dd91d6 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:04.958064 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-412fef32-a4ef-4ba4-bc27-d94d56dd91d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:04.963959 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 793/1556] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:04.966202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176b272-ec10-449e-89ac-25ff842107a7 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:04.967237 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176b272-ec10-449e-89ac-25ff842107a7 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:04.967756 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176b272-ec10-449e-89ac-25ff842107a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:04.974718 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 764/1557] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:04.974718 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412fef32-a4ef-4ba4-bc27-d94d56dd91d6 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:04.984285 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:04.989621 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:04.997650 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-5176b272-ec10-449e-89ac-25ff842107a7 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:04.998877 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-5176b272-ec10-449e-89ac-25ff842107a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:05.015207 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 794/1558] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:05.015207 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b072d97-ef34-41f1-8c66-62cfb84cb72a None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:05.015207 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b072d97-ef34-41f1-8c66-62cfb84cb72a None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:05.015207 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b072d97-ef34-41f1-8c66-62cfb84cb72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:05.033992 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-0b072d97-ef34-41f1-8c66-62cfb84cb72a None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:05.035230 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-0b072d97-ef34-41f1-8c66-62cfb84cb72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:05.043614 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 795/1559] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:05.044599 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b072d97-ef34-41f1-8c66-62cfb84cb72a None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:05.044726 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412fef32-a4ef-4ba4-bc27-d94d56dd91d6 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:05.048350 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db013a6-2e37-4799-b8d8-9c37e3250197 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:05.048998 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db013a6-2e37-4799-b8d8-9c37e3250197 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:05.049279 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db013a6-2e37-4799-b8d8-9c37e3250197 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:05.049562 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db013a6-2e37-4799-b8d8-9c37e3250197 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:05.057653 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b072d97-ef34-41f1-8c66-62cfb84cb72a None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:05.073236 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db013a6-2e37-4799-b8d8-9c37e3250197 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:05.073236 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db013a6-2e37-4799-b8d8-9c37e3250197 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:05.073460 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f017f74-3391-43be-9d42-43c176417bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:05.073460 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f017f74-3391-43be-9d42-43c176417bb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:05.073460 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f017f74-3391-43be-9d42-43c176417bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:05.073460 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f017f74-3391-43be-9d42-43c176417bb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:05.073912 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f017f74-3391-43be-9d42-43c176417bb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:05.073912 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f017f74-3391-43be-9d42-43c176417bb3 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:05.074105 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 765/1560] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:05.074105 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db013a6-2e37-4799-b8d8-9c37e3250197 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:05.078836 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 796/1561] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:05.080095 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6af572-2f14-4fbf-b5e2-f0bb2728b6f5 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:05.080369 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6af572-2f14-4fbf-b5e2-f0bb2728b6f5 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:05.080673 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6af572-2f14-4fbf-b5e2-f0bb2728b6f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:05.083524 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db013a6-2e37-4799-b8d8-9c37e3250197 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:05.088984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c86708-77c2-4541-949d-aab8d4c70fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:05.089604 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c86708-77c2-4541-949d-aab8d4c70fcf admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:05.089883 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c86708-77c2-4541-949d-aab8d4c70fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:05.090183 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c86708-77c2-4541-949d-aab8d4c70fcf admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:05.090856 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c86708-77c2-4541-949d-aab8d4c70fcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:25:05.091159 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c86708-77c2-4541-949d-aab8d4c70fcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:05.092780 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c86708-77c2-4541-949d-aab8d4c70fcf admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:05.101723 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-4e6af572-2f14-4fbf-b5e2-f0bb2728b6f5 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:05.103019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-4e6af572-2f14-4fbf-b5e2-f0bb2728b6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:05.106628 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 766/1562] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:25:05.113124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852bfca6-284b-44c1-a626-6df7c13dd19d None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:05.113124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852bfca6-284b-44c1-a626-6df7c13dd19d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:05.113124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852bfca6-284b-44c1-a626-6df7c13dd19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:05.113435 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 797/1563] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:05.113435 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde5508-0dbe-4de0-aa63-3a4dd529749d None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:05.113435 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde5508-0dbe-4de0-aa63-3a4dd529749d None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:05.113435 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde5508-0dbe-4de0-aa63-3a4dd529749d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:05.135778 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-6dde5508-0dbe-4de0-aa63-3a4dd529749d None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:05.135985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-852bfca6-284b-44c1-a626-6df7c13dd19d None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:05.136107 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-6dde5508-0dbe-4de0-aa63-3a4dd529749d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:05.136289 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-852bfca6-284b-44c1-a626-6df7c13dd19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:05.143024 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 767/1564] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:05.143160 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 798/1565] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:05.144039 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852bfca6-284b-44c1-a626-6df7c13dd19d None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:05.144367 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6d4b7-5ba8-4ff9-91dc-c1d314558fb4 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:05.144661 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6d4b7-5ba8-4ff9-91dc-c1d314558fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:05.144936 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6d4b7-5ba8-4ff9-91dc-c1d314558fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:05.156604 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852bfca6-284b-44c1-a626-6df7c13dd19d None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:05.161717 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3778b8c-6d8b-4688-9991-f64aba02c6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:05.162489 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3778b8c-6d8b-4688-9991-f64aba02c6f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:05.162639 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3778b8c-6d8b-4688-9991-f64aba02c6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:05.162907 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3778b8c-6d8b-4688-9991-f64aba02c6f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:05.163651 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3778b8c-6d8b-4688-9991-f64aba02c6f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:25:05.163987 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3778b8c-6d8b-4688-9991-f64aba02c6f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:05.165908 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3778b8c-6d8b-4688-9991-f64aba02c6f1 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:05.166642 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-8bc6d4b7-5ba8-4ff9-91dc-c1d314558fb4 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:05.167855 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-8bc6d4b7-5ba8-4ff9-91dc-c1d314558fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:05.176614 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 799/1566] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:05.182307 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc6d4b7-5ba8-4ff9-91dc-c1d314558fb4 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:05.184006 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 768/1567] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:25:05.185423 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3778b8c-6d8b-4688-9991-f64aba02c6f1 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:05.193840 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc6d4b7-5ba8-4ff9-91dc-c1d314558fb4 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:05.196990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3778b8c-6d8b-4688-9991-f64aba02c6f1 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:05.201036 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1c05d-7c68-4092-9b6b-a2397c36274f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:05.204142 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1c05d-7c68-4092-9b6b-a2397c36274f admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:05.204396 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea9d29a-c237-46f4-a5c8-a16db6c08312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:05.204552 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1c05d-7c68-4092-9b6b-a2397c36274f admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:05.204552 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1c05d-7c68-4092-9b6b-a2397c36274f admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:05.204746 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9d29a-c237-46f4-a5c8-a16db6c08312 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:05.204746 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9d29a-c237-46f4-a5c8-a16db6c08312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:05.204746 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9d29a-c237-46f4-a5c8-a16db6c08312 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:05.205024 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b1c05d-7c68-4092-9b6b-a2397c36274f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:05.205124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea9d29a-c237-46f4-a5c8-a16db6c08312 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:05.205124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: WARNING py.warnings [None req-6ea9d29a-c237-46f4-a5c8-a16db6c08312 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 14:25:05.205124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: warnings.warn(msg) Aug 05 14:25:05.205124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: Aug 05 14:25:05.206001 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea9d29a-c237-46f4-a5c8-a16db6c08312 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:05.206829 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b1c05d-7c68-4092-9b6b-a2397c36274f admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:05.218859 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 800/1568] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:05.227222 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043266e5-557c-409e-8036-262b82ba29c8 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:05.227222 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043266e5-557c-409e-8036-262b82ba29c8 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:05.227222 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043266e5-557c-409e-8036-262b82ba29c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:05.241645 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-043266e5-557c-409e-8036-262b82ba29c8 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:05.242868 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-043266e5-557c-409e-8036-262b82ba29c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:05.252499 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 801/1569] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:05.255823 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043266e5-557c-409e-8036-262b82ba29c8 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:05.262593 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043266e5-557c-409e-8036-262b82ba29c8 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:05.267138 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3be3d8-770f-42b5-8f22-fb6f3f7a8cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:05.267732 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3be3d8-770f-42b5-8f22-fb6f3f7a8cb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:05.268013 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3be3d8-770f-42b5-8f22-fb6f3f7a8cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:05.268289 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3be3d8-770f-42b5-8f22-fb6f3f7a8cb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:05.269012 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3be3d8-770f-42b5-8f22-fb6f3f7a8cb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:25:05.269316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3be3d8-770f-42b5-8f22-fb6f3f7a8cb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:05.270893 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3be3d8-770f-42b5-8f22-fb6f3f7a8cb9 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:05.282932 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 802/1570] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:25:05.284565 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3be3d8-770f-42b5-8f22-fb6f3f7a8cb9 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:05.293821 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3be3d8-770f-42b5-8f22-fb6f3f7a8cb9 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:05.298430 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc725c4a-9024-4ad3-911a-8314c83f07a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:05.298999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc725c4a-9024-4ad3-911a-8314c83f07a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:05.299283 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc725c4a-9024-4ad3-911a-8314c83f07a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:05.299553 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc725c4a-9024-4ad3-911a-8314c83f07a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:05.300223 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc725c4a-9024-4ad3-911a-8314c83f07a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:25:05.300573 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc725c4a-9024-4ad3-911a-8314c83f07a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:05.302235 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc725c4a-9024-4ad3-911a-8314c83f07a9 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:05.315154 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 803/1571] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:25:05.315154 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda0f5b-f315-4d58-b6df-ac7461f6eba6 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:05.319277 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda0f5b-f315-4d58-b6df-ac7461f6eba6 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:05.319277 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda0f5b-f315-4d58-b6df-ac7461f6eba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:05.319458 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 769/1572] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:25:05 2022] POST /identity/v3/projects => generated 375 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:05.319458 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea9d29a-c237-46f4-a5c8-a16db6c08312 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:05.343683 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-ebda0f5b-f315-4d58-b6df-ac7461f6eba6 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:05.345522 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-ebda0f5b-f315-4d58-b6df-ac7461f6eba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:05.398466 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea9d29a-c237-46f4-a5c8-a16db6c08312 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:05.406325 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403b28c-b49b-47e9-be13-5d02e8deb997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:05.406325 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403b28c-b49b-47e9-be13-5d02e8deb997 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:05.406325 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403b28c-b49b-47e9-be13-5d02e8deb997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:05.406325 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403b28c-b49b-47e9-be13-5d02e8deb997 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:05.406983 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c403b28c-b49b-47e9-be13-5d02e8deb997 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:05.408162 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c403b28c-b49b-47e9-be13-5d02e8deb997 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:05.417534 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 804/1573] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:05.418365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda0f5b-f315-4d58-b6df-ac7461f6eba6 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:05.429362 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda0f5b-f315-4d58-b6df-ac7461f6eba6 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:05.434159 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c9010c-7b05-4114-9343-60c5329065ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:05.434700 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c9010c-7b05-4114-9343-60c5329065ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:05.434976 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c9010c-7b05-4114-9343-60c5329065ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:05.435247 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c9010c-7b05-4114-9343-60c5329065ea admin admin] PATH_INFO: `/v3/projects/a41a81a283c744c2bd3276e1caa5d619/users/a9212230bba44398af36ffdee8a0d62d/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:05.451053 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c9010c-7b05-4114-9343-60c5329065ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=a41a81a283c744c2bd3276e1caa5d619, user_id=a9212230bba44398af36ffdee8a0d62d, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:05.453789 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c9010c-7b05-4114-9343-60c5329065ea admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:05.518512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 770/1574] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:25:05 2022] POST /identity/v3/projects => generated 371 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:05.519487 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403b28c-b49b-47e9-be13-5d02e8deb997 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:05.559511 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 805/1575] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:25:05 2022] PUT /identity/v3/projects/a41a81a283c744c2bd3276e1caa5d619/users/a9212230bba44398af36ffdee8a0d62d/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:05.559511 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c9010c-7b05-4114-9343-60c5329065ea admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:05.566111 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:05.587845 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403b28c-b49b-47e9-be13-5d02e8deb997 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:05.628287 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:05.640648 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c9010c-7b05-4114-9343-60c5329065ea admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:05.645906 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf0ca4-cfe5-4588-8287-d1c1ac6ae90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:05.647205 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf0ca4-cfe5-4588-8287-d1c1ac6ae90c admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:05.647429 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf0ca4-cfe5-4588-8287-d1c1ac6ae90c admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:05.647669 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf0ca4-cfe5-4588-8287-d1c1ac6ae90c admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:05.648330 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf0ca4-cfe5-4588-8287-d1c1ac6ae90c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:05.650398 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf0ca4-cfe5-4588-8287-d1c1ac6ae90c admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:05.657310 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6502653-b4d2-4328-aa45-a5420deffa7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:05.657531 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6502653-b4d2-4328-aa45-a5420deffa7b admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:05.657809 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6502653-b4d2-4328-aa45-a5420deffa7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:05.658078 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6502653-b4d2-4328-aa45-a5420deffa7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:05.658953 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6502653-b4d2-4328-aa45-a5420deffa7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:25:05.659259 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6502653-b4d2-4328-aa45-a5420deffa7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:05.660901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6502653-b4d2-4328-aa45-a5420deffa7b admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:05.677541 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 771/1576] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:25:05.681392 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6502653-b4d2-4328-aa45-a5420deffa7b admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:05.692329 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6502653-b4d2-4328-aa45-a5420deffa7b admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:05.698152 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4f019-7715-4cc7-9ed0-efd4f264fb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:05.698771 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4f019-7715-4cc7-9ed0-efd4f264fb68 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:05.699067 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4f019-7715-4cc7-9ed0-efd4f264fb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:05.699328 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4f019-7715-4cc7-9ed0-efd4f264fb68 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:05.699987 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc4f019-7715-4cc7-9ed0-efd4f264fb68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:05.701928 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc4f019-7715-4cc7-9ed0-efd4f264fb68 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:05.745651 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 806/1577] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:25:05 2022] POST /identity/v3/projects => generated 365 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:05.745651 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf0ca4-cfe5-4588-8287-d1c1ac6ae90c admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:05.787848 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 772/1578] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:25:05 2022] POST /identity/v3/projects => generated 389 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:05.788705 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4f019-7715-4cc7-9ed0-efd4f264fb68 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:05.804670 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf0ca4-cfe5-4588-8287-d1c1ac6ae90c admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:05.861748 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4f019-7715-4cc7-9ed0-efd4f264fb68 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:05.867645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347f8b0-5b54-41a1-957f-64ed1a3f0cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:05.868355 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347f8b0-5b54-41a1-957f-64ed1a3f0cb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:05.868621 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347f8b0-5b54-41a1-957f-64ed1a3f0cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:05.868896 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347f8b0-5b54-41a1-957f-64ed1a3f0cb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:05.869623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347f8b0-5b54-41a1-957f-64ed1a3f0cb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:25:05.869928 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ccc6db-6038-4a97-aab9-5115d8036086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:05.870425 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347f8b0-5b54-41a1-957f-64ed1a3f0cb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:05.870787 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ccc6db-6038-4a97-aab9-5115d8036086 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:05.871069 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ccc6db-6038-4a97-aab9-5115d8036086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:05.871341 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ccc6db-6038-4a97-aab9-5115d8036086 admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:05.871547 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347f8b0-5b54-41a1-957f-64ed1a3f0cb6 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:05.872162 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ccc6db-6038-4a97-aab9-5115d8036086 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:05.873314 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: WARNING py.warnings [None req-38ccc6db-6038-4a97-aab9-5115d8036086 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 14:25:05.873314 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: warnings.warn(msg) Aug 05 14:25:05.873314 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: Aug 05 14:25:05.874625 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ccc6db-6038-4a97-aab9-5115d8036086 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:05.885150 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 773/1579] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:25:05.886165 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347f8b0-5b54-41a1-957f-64ed1a3f0cb6 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:05.907348 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347f8b0-5b54-41a1-957f-64ed1a3f0cb6 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:05.907348 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce56400-76f8-4a7f-bba4-c74ced23ca3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:05.907792 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce56400-76f8-4a7f-bba4-c74ced23ca3d admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:05.908072 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce56400-76f8-4a7f-bba4-c74ced23ca3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:05.908343 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce56400-76f8-4a7f-bba4-c74ced23ca3d admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:05.909030 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce56400-76f8-4a7f-bba4-c74ced23ca3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:05.910942 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce56400-76f8-4a7f-bba4-c74ced23ca3d admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:05.962118 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 807/1580] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:25:05 2022] POST /identity/v3/users => generated 288 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:05.963662 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b117459-1ab7-4117-8869-ca767db1b50b None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:05.964462 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b117459-1ab7-4117-8869-ca767db1b50b None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:05.964462 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b117459-1ab7-4117-8869-ca767db1b50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:05.990166 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 774/1581] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:25:05 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:05.998200 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce56400-76f8-4a7f-bba4-c74ced23ca3d admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:05.998338 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-0b117459-1ab7-4117-8869-ca767db1b50b None None] MFA Rules not processed for user `a9212230bba44398af36ffdee8a0d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:05.998338 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-0b117459-1ab7-4117-8869-ca767db1b50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:06.002213 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce56400-76f8-4a7f-bba4-c74ced23ca3d admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:06.007094 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6803a38a-f910-458e-adff-d5cb5aa756d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:06.007696 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803a38a-f910-458e-adff-d5cb5aa756d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:06.007973 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803a38a-f910-458e-adff-d5cb5aa756d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:06.008306 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803a38a-f910-458e-adff-d5cb5aa756d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:06.008916 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6803a38a-f910-458e-adff-d5cb5aa756d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:06.010815 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6803a38a-f910-458e-adff-d5cb5aa756d3 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:06.062021 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 808/1582] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:06.062936 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b117459-1ab7-4117-8869-ca767db1b50b None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:06.070908 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:06.076485 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b117459-1ab7-4117-8869-ca767db1b50b None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:06.081092 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9124f-f607-492e-9b28-1ec2e7c34028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:06.081707 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9124f-f607-492e-9b28-1ec2e7c34028 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:06.081989 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9124f-f607-492e-9b28-1ec2e7c34028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:06.082263 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9124f-f607-492e-9b28-1ec2e7c34028 admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:06.082928 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee9124f-f607-492e-9b28-1ec2e7c34028 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:06.085441 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee9124f-f607-492e-9b28-1ec2e7c34028 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:06.105769 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 775/1583] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:25:05 2022] POST /identity/v3/projects => generated 375 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:06.105769 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6803a38a-f910-458e-adff-d5cb5aa756d3 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:06.118153 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:06.167057 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6803a38a-f910-458e-adff-d5cb5aa756d3 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:06.171636 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6089082-0117-4256-8740-d274e125b2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:06.172250 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6089082-0117-4256-8740-d274e125b2c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:06.172581 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6089082-0117-4256-8740-d274e125b2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:06.172860 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6089082-0117-4256-8740-d274e125b2c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:06.173544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6089082-0117-4256-8740-d274e125b2c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:06.175450 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6089082-0117-4256-8740-d274e125b2c4 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:06.293806 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 809/1584] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:25:06 2022] POST /identity/v3/users => generated 283 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:06.293806 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9124f-f607-492e-9b28-1ec2e7c34028 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:06.293806 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9124f-f607-492e-9b28-1ec2e7c34028 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:06.293806 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf979876-d9bd-4934-9404-fbe0df8a75e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:06.294659 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf979876-d9bd-4934-9404-fbe0df8a75e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:06.294659 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf979876-d9bd-4934-9404-fbe0df8a75e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:06.294659 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf979876-d9bd-4934-9404-fbe0df8a75e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:06.294659 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf979876-d9bd-4934-9404-fbe0df8a75e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:06.294659 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf979876-d9bd-4934-9404-fbe0df8a75e6 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:06.295110 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 776/1585] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:25:06 2022] POST /identity/v3/users => generated 295 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:06.295110 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6089082-0117-4256-8740-d274e125b2c4 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:06.295110 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6089082-0117-4256-8740-d274e125b2c4 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:06.295110 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a5755-2e12-4e17-b0d3-750abe1193eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:06.295466 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a5755-2e12-4e17-b0d3-750abe1193eb admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:06.295466 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a5755-2e12-4e17-b0d3-750abe1193eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:06.295466 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a5755-2e12-4e17-b0d3-750abe1193eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:06.295466 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a5755-2e12-4e17-b0d3-750abe1193eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:06.295901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a5755-2e12-4e17-b0d3-750abe1193eb admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:06.306385 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 777/1586] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:06.306979 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a5755-2e12-4e17-b0d3-750abe1193eb admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:06.317369 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a5755-2e12-4e17-b0d3-750abe1193eb admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:06.322958 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2558699d-fbf6-4f45-86cc-0c9b4d851911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:06.323546 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2558699d-fbf6-4f45-86cc-0c9b4d851911 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:06.323819 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2558699d-fbf6-4f45-86cc-0c9b4d851911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:06.324119 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2558699d-fbf6-4f45-86cc-0c9b4d851911 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:06.324952 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2558699d-fbf6-4f45-86cc-0c9b4d851911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:06.326587 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2558699d-fbf6-4f45-86cc-0c9b4d851911 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:06.338610 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 778/1587] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:06.339386 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2558699d-fbf6-4f45-86cc-0c9b4d851911 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:06.349772 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2558699d-fbf6-4f45-86cc-0c9b4d851911 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:06.354778 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcbd578-3fee-483f-b500-1d3378cca6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:06.355405 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbd578-3fee-483f-b500-1d3378cca6a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:06.355677 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbd578-3fee-483f-b500-1d3378cca6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:06.355944 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbd578-3fee-483f-b500-1d3378cca6a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:06.356624 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcbd578-3fee-483f-b500-1d3378cca6a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:06.358534 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcbd578-3fee-483f-b500-1d3378cca6a4 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:06.393944 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 810/1588] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:25:06 2022] POST /identity/v3/projects => generated 383 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:06.394817 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf979876-d9bd-4934-9404-fbe0df8a75e6 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:06.464628 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 779/1589] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:25:06 2022] POST /identity/v3/projects => generated 369 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:06.469947 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcbd578-3fee-483f-b500-1d3378cca6a4 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:06.471439 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf979876-d9bd-4934-9404-fbe0df8a75e6 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:06.564294 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f03b5e-8243-466d-900d-9ffed0217c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:06.565359 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f03b5e-8243-466d-900d-9ffed0217c13 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:06.566151 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f03b5e-8243-466d-900d-9ffed0217c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:06.566710 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f03b5e-8243-466d-900d-9ffed0217c13 admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:06.570121 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f03b5e-8243-466d-900d-9ffed0217c13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:06.572608 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f03b5e-8243-466d-900d-9ffed0217c13 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:06.593507 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcbd578-3fee-483f-b500-1d3378cca6a4 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:06.598799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edfc6cc-c6c1-4031-9412-5a8aa905215a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:06.601500 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edfc6cc-c6c1-4031-9412-5a8aa905215a admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:06.601500 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edfc6cc-c6c1-4031-9412-5a8aa905215a admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:06.601500 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edfc6cc-c6c1-4031-9412-5a8aa905215a admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:06.603189 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edfc6cc-c6c1-4031-9412-5a8aa905215a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:06.611135 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edfc6cc-c6c1-4031-9412-5a8aa905215a admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:06.616465 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 780/1590] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:06.617272 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edfc6cc-c6c1-4031-9412-5a8aa905215a admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:06.632360 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edfc6cc-c6c1-4031-9412-5a8aa905215a admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:06.638555 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b32dbd-c978-4482-8cf1-2289f9daad6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:06.644585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b32dbd-c978-4482-8cf1-2289f9daad6c admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:06.644585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b32dbd-c978-4482-8cf1-2289f9daad6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:06.644585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b32dbd-c978-4482-8cf1-2289f9daad6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:06.644585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b32dbd-c978-4482-8cf1-2289f9daad6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:06.644585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b32dbd-c978-4482-8cf1-2289f9daad6c admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:06.655935 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 781/1591] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:06.656905 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b32dbd-c978-4482-8cf1-2289f9daad6c admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:06.674210 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b32dbd-c978-4482-8cf1-2289f9daad6c admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:06.681908 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e4814-318b-4b58-894b-be3c444cdbd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:06.682625 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e4814-318b-4b58-894b-be3c444cdbd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:06.682971 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e4814-318b-4b58-894b-be3c444cdbd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:06.683313 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e4814-318b-4b58-894b-be3c444cdbd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:06.684206 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2e4814-318b-4b58-894b-be3c444cdbd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:06.686529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2e4814-318b-4b58-894b-be3c444cdbd8 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:06.706535 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 782/1592] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:06.708271 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e4814-318b-4b58-894b-be3c444cdbd8 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:06.713493 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 811/1593] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:25:06 2022] POST /identity/v3/users => generated 288 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:06.714783 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f03b5e-8243-466d-900d-9ffed0217c13 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:06.722927 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e4814-318b-4b58-894b-be3c444cdbd8 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:06.726582 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f03b5e-8243-466d-900d-9ffed0217c13 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:06.731773 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bbad3b-6b3d-43c6-986d-5d03251c5b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:06.732289 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afceccd5-7fbf-480f-b088-f0edcc469528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:06.732597 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bbad3b-6b3d-43c6-986d-5d03251c5b1a admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:06.732927 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bbad3b-6b3d-43c6-986d-5d03251c5b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:06.733264 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bbad3b-6b3d-43c6-986d-5d03251c5b1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:06.733955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afceccd5-7fbf-480f-b088-f0edcc469528 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:06.734074 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bbad3b-6b3d-43c6-986d-5d03251c5b1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:06.734472 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afceccd5-7fbf-480f-b088-f0edcc469528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:06.734993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afceccd5-7fbf-480f-b088-f0edcc469528 admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:06.735802 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afceccd5-7fbf-480f-b088-f0edcc469528 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:06.736038 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bbad3b-6b3d-43c6-986d-5d03251c5b1a admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:06.738206 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afceccd5-7fbf-480f-b088-f0edcc469528 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:06.759658 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 783/1594] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:06.760591 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bbad3b-6b3d-43c6-986d-5d03251c5b1a admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:06.773403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bbad3b-6b3d-43c6-986d-5d03251c5b1a admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:06.778483 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30197217-1711-45ac-87cf-cb715a88ab4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:06.779159 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30197217-1711-45ac-87cf-cb715a88ab4b admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:06.779504 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30197217-1711-45ac-87cf-cb715a88ab4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:06.779846 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30197217-1711-45ac-87cf-cb715a88ab4b admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:06.780591 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30197217-1711-45ac-87cf-cb715a88ab4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:06.782574 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30197217-1711-45ac-87cf-cb715a88ab4b admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:06.854818 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 812/1595] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:25:06 2022] POST /identity/v3/users => generated 292 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:06.855660 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afceccd5-7fbf-480f-b088-f0edcc469528 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:06.867183 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afceccd5-7fbf-480f-b088-f0edcc469528 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:06.872893 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f923a-aa90-4a4e-b304-261983e04a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:06.873601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f923a-aa90-4a4e-b304-261983e04a32 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:06.874042 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f923a-aa90-4a4e-b304-261983e04a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:06.874473 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f923a-aa90-4a4e-b304-261983e04a32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:06.875172 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021f923a-aa90-4a4e-b304-261983e04a32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:06.876785 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021f923a-aa90-4a4e-b304-261983e04a32 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:06.882450 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 784/1596] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:25:06 2022] POST /identity/v3/users => generated 291 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:06.883459 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30197217-1711-45ac-87cf-cb715a88ab4b admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:06.890229 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:06.894118 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 813/1597] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:06.894118 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f923a-aa90-4a4e-b304-261983e04a32 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:06.894508 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30197217-1711-45ac-87cf-cb715a88ab4b admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:06.900521 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4422a07-1796-4f93-847c-056f3cc48051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:06.901220 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4422a07-1796-4f93-847c-056f3cc48051 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:06.901568 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4422a07-1796-4f93-847c-056f3cc48051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:06.901932 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4422a07-1796-4f93-847c-056f3cc48051 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:06.902701 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4422a07-1796-4f93-847c-056f3cc48051 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:06.903181 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f923a-aa90-4a4e-b304-261983e04a32 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:06.904433 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4422a07-1796-4f93-847c-056f3cc48051 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:06.907985 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c00f28-732f-494e-8890-3b03af204997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:06.908690 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c00f28-732f-494e-8890-3b03af204997 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:06.908977 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c00f28-732f-494e-8890-3b03af204997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:06.909247 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c00f28-732f-494e-8890-3b03af204997 admin admin] PATH_INFO: `/v3/projects/85f448f6bc5845d1b0567924c78a6162/users/3bbe964ead724a4da59620786b5706d8/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:06.917490 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 785/1598] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:06.921722 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4422a07-1796-4f93-847c-056f3cc48051 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:06.926059 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c00f28-732f-494e-8890-3b03af204997 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85f448f6bc5845d1b0567924c78a6162, user_id=3bbe964ead724a4da59620786b5706d8, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:06.929551 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c00f28-732f-494e-8890-3b03af204997 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:06.933649 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4422a07-1796-4f93-847c-056f3cc48051 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:06.938784 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe80dc7-0ea8-4813-a9a1-06b1f3692ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:06.939370 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe80dc7-0ea8-4813-a9a1-06b1f3692ee9 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:06.939650 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe80dc7-0ea8-4813-a9a1-06b1f3692ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:06.939930 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe80dc7-0ea8-4813-a9a1-06b1f3692ee9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:06.940664 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe80dc7-0ea8-4813-a9a1-06b1f3692ee9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:06.942279 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe80dc7-0ea8-4813-a9a1-06b1f3692ee9 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:06.954712 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 786/1599] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:06.955314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe80dc7-0ea8-4813-a9a1-06b1f3692ee9 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:06.966802 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe80dc7-0ea8-4813-a9a1-06b1f3692ee9 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:06.972442 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e095d-4f57-44dc-ade5-a09559c47db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:06.973141 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e095d-4f57-44dc-ade5-a09559c47db8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:06.973422 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e095d-4f57-44dc-ade5-a09559c47db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:06.973697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e095d-4f57-44dc-ade5-a09559c47db8 admin admin] PATH_INFO: `/v3/projects/dbe84e660595496fbff4fb67c998c5f3/users/c6da0409312749478378d65d478265ba/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:06.996840 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e095d-4f57-44dc-ade5-a09559c47db8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbe84e660595496fbff4fb67c998c5f3, user_id=c6da0409312749478378d65d478265ba, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:06.996840 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e095d-4f57-44dc-ade5-a09559c47db8 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:07.053254 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 814/1600] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:25:06 2022] PUT /identity/v3/projects/85f448f6bc5845d1b0567924c78a6162/users/3bbe964ead724a4da59620786b5706d8/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:07.054288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c00f28-732f-494e-8890-3b03af204997 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:07.066692 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:07.086823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 787/1601] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:25:06 2022] PUT /identity/v3/projects/dbe84e660595496fbff4fb67c998c5f3/users/c6da0409312749478378d65d478265ba/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:07.086823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e095d-4f57-44dc-ade5-a09559c47db8 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:07.151824 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c00f28-732f-494e-8890-3b03af204997 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:07.173528 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e095d-4f57-44dc-ade5-a09559c47db8 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:07.180809 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7dc13-ab0b-4a78-9b2c-1a7088ab8890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:07.181508 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7dc13-ab0b-4a78-9b2c-1a7088ab8890 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:07.181712 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7dc13-ab0b-4a78-9b2c-1a7088ab8890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:07.181984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7dc13-ab0b-4a78-9b2c-1a7088ab8890 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:07.182723 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7dc13-ab0b-4a78-9b2c-1a7088ab8890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:07.184352 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7dc13-ab0b-4a78-9b2c-1a7088ab8890 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:07.195845 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 788/1602] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:07.196671 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7dc13-ab0b-4a78-9b2c-1a7088ab8890 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:07.207498 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7dc13-ab0b-4a78-9b2c-1a7088ab8890 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:07.214091 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7e393-07f7-47de-bcc0-49cc45304582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:07.215266 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7e393-07f7-47de-bcc0-49cc45304582 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:07.215569 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7e393-07f7-47de-bcc0-49cc45304582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:07.215912 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7e393-07f7-47de-bcc0-49cc45304582 admin admin] PATH_INFO: `/v3/projects/8ba19526865c4d699b2521c3fd5b2847/users/2870756169cd4516bf733bc2f257186c/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:07.219621 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a101c-1386-4c09-a262-23135ee83e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:07.220254 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a101c-1386-4c09-a262-23135ee83e29 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:07.220586 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a101c-1386-4c09-a262-23135ee83e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:07.220848 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a101c-1386-4c09-a262-23135ee83e29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:07.221725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a101c-1386-4c09-a262-23135ee83e29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:07.223317 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a101c-1386-4c09-a262-23135ee83e29 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:07.234068 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7e393-07f7-47de-bcc0-49cc45304582 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ba19526865c4d699b2521c3fd5b2847, user_id=2870756169cd4516bf733bc2f257186c, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:07.237329 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 815/1603] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:07.237329 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a101c-1386-4c09-a262-23135ee83e29 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:07.241225 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7e393-07f7-47de-bcc0-49cc45304582 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:07.247269 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a101c-1386-4c09-a262-23135ee83e29 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:07.251940 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de13fa06-6d14-4881-a977-3a4337dc9a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:07.252689 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13fa06-6d14-4881-a977-3a4337dc9a52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:07.252971 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13fa06-6d14-4881-a977-3a4337dc9a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:07.253245 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13fa06-6d14-4881-a977-3a4337dc9a52 admin admin] PATH_INFO: `/v3/projects/5c13896c64e44b60838d3f313407433f/users/084b18670c2a4c1396d5912d130b2bac/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:07.272249 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de13fa06-6d14-4881-a977-3a4337dc9a52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c13896c64e44b60838d3f313407433f, user_id=084b18670c2a4c1396d5912d130b2bac, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:07.275162 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de13fa06-6d14-4881-a977-3a4337dc9a52 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:07.336691 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 816/1604] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:25:07 2022] PUT /identity/v3/projects/5c13896c64e44b60838d3f313407433f/users/084b18670c2a4c1396d5912d130b2bac/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:07.337676 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 789/1605] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:25:07 2022] PUT /identity/v3/projects/8ba19526865c4d699b2521c3fd5b2847/users/2870756169cd4516bf733bc2f257186c/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:07.337812 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de13fa06-6d14-4881-a977-3a4337dc9a52 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:07.349838 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401737f8-395c-4293-b168-3df245ef1db4 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:07.353108 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401737f8-395c-4293-b168-3df245ef1db4 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:07.353108 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401737f8-395c-4293-b168-3df245ef1db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:07.379423 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-401737f8-395c-4293-b168-3df245ef1db4 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:07.381936 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-401737f8-395c-4293-b168-3df245ef1db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:07.428402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de13fa06-6d14-4881-a977-3a4337dc9a52 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:07.434319 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4deb170-e67d-4a6c-9cd9-5bcf17f0de56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:07.435023 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4deb170-e67d-4a6c-9cd9-5bcf17f0de56 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:07.435362 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4deb170-e67d-4a6c-9cd9-5bcf17f0de56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:07.435700 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4deb170-e67d-4a6c-9cd9-5bcf17f0de56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:07.436497 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4deb170-e67d-4a6c-9cd9-5bcf17f0de56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:07.438680 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4deb170-e67d-4a6c-9cd9-5bcf17f0de56 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:07.455213 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 817/1606] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:07.457361 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f10595-3ec8-4fd4-80df-c47e7975c5af None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:07.458150 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f10595-3ec8-4fd4-80df-c47e7975c5af None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:07.458433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f10595-3ec8-4fd4-80df-c47e7975c5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:07.477772 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 790/1607] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:07.478767 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401737f8-395c-4293-b168-3df245ef1db4 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:07.490206 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401737f8-395c-4293-b168-3df245ef1db4 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:07.500214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-44f10595-3ec8-4fd4-80df-c47e7975c5af None None] MFA Rules not processed for user `c6da0409312749478378d65d478265ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:07.501336 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a86d4c6-479e-448f-9afe-eefa55d8eeb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:07.501772 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-44f10595-3ec8-4fd4-80df-c47e7975c5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:07.502032 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a86d4c6-479e-448f-9afe-eefa55d8eeb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:07.502373 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a86d4c6-479e-448f-9afe-eefa55d8eeb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:07.502656 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a86d4c6-479e-448f-9afe-eefa55d8eeb2 admin admin] PATH_INFO: `/v3/projects/84346e3c882347dfb36ba372631d5e35/users/6da90312772a4cd6976146c834b993fc/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:07.525021 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:07.529581 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a86d4c6-479e-448f-9afe-eefa55d8eeb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84346e3c882347dfb36ba372631d5e35, user_id=6da90312772a4cd6976146c834b993fc, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:07.534357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a86d4c6-479e-448f-9afe-eefa55d8eeb2 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:07.642429 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 818/1608] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:07.644048 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f10595-3ec8-4fd4-80df-c47e7975c5af None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:07.655501 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 791/1609] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:25:07 2022] PUT /identity/v3/projects/84346e3c882347dfb36ba372631d5e35/users/6da90312772a4cd6976146c834b993fc/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:07.658615 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b2fe3-a662-459b-9c6a-f5622e7e9f35 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:07.658615 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b2fe3-a662-459b-9c6a-f5622e7e9f35 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:07.658615 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b2fe3-a662-459b-9c6a-f5622e7e9f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:07.660129 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f10595-3ec8-4fd4-80df-c47e7975c5af None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:07.691076 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:07.703669 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-da9b2fe3-a662-459b-9c6a-f5622e7e9f35 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:07.703669 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-da9b2fe3-a662-459b-9c6a-f5622e7e9f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:07.759304 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f20ae5-687f-4646-a2aa-b4ec78bb3ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:07.759891 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f20ae5-687f-4646-a2aa-b4ec78bb3ed6 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:07.760243 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f20ae5-687f-4646-a2aa-b4ec78bb3ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:07.760620 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f20ae5-687f-4646-a2aa-b4ec78bb3ed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:07.761436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f20ae5-687f-4646-a2aa-b4ec78bb3ed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:07.763304 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f20ae5-687f-4646-a2aa-b4ec78bb3ed6 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:07.774391 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 819/1610] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:07.775767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4f8a9-fdd3-4f14-a45a-9fdb81530415 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:07.776064 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4f8a9-fdd3-4f14-a45a-9fdb81530415 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:07.776576 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4f8a9-fdd3-4f14-a45a-9fdb81530415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:07.798696 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 792/1611] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:07.799518 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9b2fe3-a662-459b-9c6a-f5622e7e9f35 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:07.810819 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9b2fe3-a662-459b-9c6a-f5622e7e9f35 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:07.819996 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35bc21c-65ba-498b-8869-e92bedfe8b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:07.820840 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bc21c-65ba-498b-8869-e92bedfe8b7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:07.821224 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bc21c-65ba-498b-8869-e92bedfe8b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:07.821562 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bc21c-65ba-498b-8869-e92bedfe8b7b admin admin] PATH_INFO: `/v3/projects/0b15967673f14cf9a208a6143a4e87cc/users/99dc94fff06f42c39f3750484f6d8e9d/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:07.824883 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-f8c4f8a9-fdd3-4f14-a45a-9fdb81530415 None None] MFA Rules not processed for user `2870756169cd4516bf733bc2f257186c`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:07.826422 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-f8c4f8a9-fdd3-4f14-a45a-9fdb81530415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:07.848062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35bc21c-65ba-498b-8869-e92bedfe8b7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b15967673f14cf9a208a6143a4e87cc, user_id=99dc94fff06f42c39f3750484f6d8e9d, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:07.851515 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35bc21c-65ba-498b-8869-e92bedfe8b7b admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:07.926708 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 820/1612] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:07.929671 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4f8a9-fdd3-4f14-a45a-9fdb81530415 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:07.931344 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-f8c4f8a9-fdd3-4f14-a45a-9fdb81530415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:07.933857 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 793/1613] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:25:07 2022] PUT /identity/v3/projects/0b15967673f14cf9a208a6143a4e87cc/users/99dc94fff06f42c39f3750484f6d8e9d/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:07.935054 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35bc21c-65ba-498b-8869-e92bedfe8b7b admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:07.987804 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:08.004384 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4f8a9-fdd3-4f14-a45a-9fdb81530415 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:08.022220 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35bc21c-65ba-498b-8869-e92bedfe8b7b admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:08.029165 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3b1d6-6c46-4679-ac75-e7c29118620b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:08.033616 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3b1d6-6c46-4679-ac75-e7c29118620b admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:08.033616 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3b1d6-6c46-4679-ac75-e7c29118620b admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:08.033616 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3b1d6-6c46-4679-ac75-e7c29118620b admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:08.033616 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f3b1d6-6c46-4679-ac75-e7c29118620b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:08.033616 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f3b1d6-6c46-4679-ac75-e7c29118620b admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:08.054325 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea3e7de-f607-4497-adf4-22bbba323ad0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:08.055116 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3e7de-f607-4497-adf4-22bbba323ad0 service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:08.055445 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3e7de-f607-4497-adf4-22bbba323ad0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:08.055791 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3e7de-f607-4497-adf4-22bbba323ad0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:08.065884 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea3e7de-f607-4497-adf4-22bbba323ad0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:08.067373 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea3e7de-f607-4497-adf4-22bbba323ad0 service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:08.078776 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 821/1614] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:08.079770 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea3e7de-f607-4497-adf4-22bbba323ad0 service neutron] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:08.093595 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea3e7de-f607-4497-adf4-22bbba323ad0 service neutron] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:08.102001 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b79b00-59de-417e-8f10-fb12973da08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:08.103331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b79b00-59de-417e-8f10-fb12973da08a admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:08.104325 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b79b00-59de-417e-8f10-fb12973da08a admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:08.105208 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b79b00-59de-417e-8f10-fb12973da08a admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:08.106449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b79b00-59de-417e-8f10-fb12973da08a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:08.109404 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b79b00-59de-417e-8f10-fb12973da08a admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:08.128012 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 822/1615] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:08.129328 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b79b00-59de-417e-8f10-fb12973da08a admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:08.161350 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 794/1616] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:25:07 2022] POST /identity/v3/projects => generated 371 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:08.161350 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3b1d6-6c46-4679-ac75-e7c29118620b admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:08.161593 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b79b00-59de-417e-8f10-fb12973da08a admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:08.240462 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3b1d6-6c46-4679-ac75-e7c29118620b admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:08.249853 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c7119-443c-4ec8-8f81-24a634a9d13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:08.250749 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c7119-443c-4ec8-8f81-24a634a9d13e admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:08.251310 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c7119-443c-4ec8-8f81-24a634a9d13e admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:08.251815 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c7119-443c-4ec8-8f81-24a634a9d13e admin admin] PATH_INFO: `/v3/projects/4de55490e2a84e4ba145a09dd6c9a877/users/b5c7a069800d476d9df8f945120ad7ea/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:08.253789 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a459c5eb-f655-4576-89bb-02fad49ca9d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:08.254438 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459c5eb-f655-4576-89bb-02fad49ca9d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:08.254713 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459c5eb-f655-4576-89bb-02fad49ca9d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:08.254986 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459c5eb-f655-4576-89bb-02fad49ca9d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:08.270952 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272c7119-443c-4ec8-8f81-24a634a9d13e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4de55490e2a84e4ba145a09dd6c9a877, user_id=b5c7a069800d476d9df8f945120ad7ea, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:08.274799 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272c7119-443c-4ec8-8f81-24a634a9d13e admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:08.275548 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a459c5eb-f655-4576-89bb-02fad49ca9d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:08.277738 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a459c5eb-f655-4576-89bb-02fad49ca9d0 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:08.296186 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 795/1617] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:08.297984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a459c5eb-f655-4576-89bb-02fad49ca9d0 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:08.314535 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a459c5eb-f655-4576-89bb-02fad49ca9d0 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:08.324359 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e0d4d4-3edd-44b0-a4c2-47f0ef2353ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:08.325565 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0d4d4-3edd-44b0-a4c2-47f0ef2353ed admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:08.325565 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0d4d4-3edd-44b0-a4c2-47f0ef2353ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:08.325857 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0d4d4-3edd-44b0-a4c2-47f0ef2353ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:08.327005 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e0d4d4-3edd-44b0-a4c2-47f0ef2353ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:08.329374 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e0d4d4-3edd-44b0-a4c2-47f0ef2353ed admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:08.341720 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:08.386148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 823/1618] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:25:08 2022] PUT /identity/v3/projects/4de55490e2a84e4ba145a09dd6c9a877/users/b5c7a069800d476d9df8f945120ad7ea/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:08.387508 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb3e2a-2906-4678-b8d9-91992efaf1cf None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:08.387927 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb3e2a-2906-4678-b8d9-91992efaf1cf None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:08.388207 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb3e2a-2906-4678-b8d9-91992efaf1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:08.410583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-d9bb3e2a-2906-4678-b8d9-91992efaf1cf None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:08.411815 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-d9bb3e2a-2906-4678-b8d9-91992efaf1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:08.427169 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 796/1619] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:25:08 2022] POST /identity/v3/projects => generated 365 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:08.427169 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e0d4d4-3edd-44b0-a4c2-47f0ef2353ed admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:08.523521 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e0d4d4-3edd-44b0-a4c2-47f0ef2353ed admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:08.530756 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4988d1-bdac-42ba-9384-5c164e2b0560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:08.532465 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4988d1-bdac-42ba-9384-5c164e2b0560 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:08.532465 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4988d1-bdac-42ba-9384-5c164e2b0560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:08.532465 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4988d1-bdac-42ba-9384-5c164e2b0560 admin admin] PATH_INFO: `/v3/projects/84346e3c882347dfb36ba372631d5e35/users/6da90312772a4cd6976146c834b993fc/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:08.534665 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4988d1-bdac-42ba-9384-5c164e2b0560 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84346e3c882347dfb36ba372631d5e35, user_id=6da90312772a4cd6976146c834b993fc, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:08.538533 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4988d1-bdac-42ba-9384-5c164e2b0560 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:08.554570 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 824/1620] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:08.554570 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb3e2a-2906-4678-b8d9-91992efaf1cf None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:08.554881 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 797/1621] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:25:08 2022] PUT /identity/v3/projects/84346e3c882347dfb36ba372631d5e35/users/6da90312772a4cd6976146c834b993fc/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:08.555682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911d197-6cc1-4db1-9146-0a9166e674a9 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:08.555974 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911d197-6cc1-4db1-9146-0a9166e674a9 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:08.556252 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911d197-6cc1-4db1-9146-0a9166e674a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:08.570171 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb3e2a-2906-4678-b8d9-91992efaf1cf None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:08.585814 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:08.597830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-7911d197-6cc1-4db1-9146-0a9166e674a9 None None] MFA Rules not processed for user `b5c7a069800d476d9df8f945120ad7ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:08.600999 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-7911d197-6cc1-4db1-9146-0a9166e674a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:08.672508 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210de2df-e938-4b30-b513-1a47e335d3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:08.673534 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210de2df-e938-4b30-b513-1a47e335d3a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:08.673826 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210de2df-e938-4b30-b513-1a47e335d3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:08.674191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210de2df-e938-4b30-b513-1a47e335d3a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:08.674949 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210de2df-e938-4b30-b513-1a47e335d3a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:08.678079 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210de2df-e938-4b30-b513-1a47e335d3a4 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:08.700643 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 798/1622] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:08.701830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7911d197-6cc1-4db1-9146-0a9166e674a9 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:08.722550 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7911d197-6cc1-4db1-9146-0a9166e674a9 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:08.729700 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6372a0-3b35-4b67-a485-41d695bae8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:08.730334 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6372a0-3b35-4b67-a485-41d695bae8e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:08.730612 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6372a0-3b35-4b67-a485-41d695bae8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:08.730884 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6372a0-3b35-4b67-a485-41d695bae8e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:08.731741 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6372a0-3b35-4b67-a485-41d695bae8e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:08.734111 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6372a0-3b35-4b67-a485-41d695bae8e4 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:08.868456 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 825/1623] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:25:08 2022] POST /identity/v3/users => generated 292 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:08.869957 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ac9cd-5f17-4ed1-8cbe-6782cbb3b0ed None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:08.870214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ac9cd-5f17-4ed1-8cbe-6782cbb3b0ed None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:08.870474 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ac9cd-5f17-4ed1-8cbe-6782cbb3b0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:08.883386 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 799/1624] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:25:08 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:08.883884 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6372a0-3b35-4b67-a485-41d695bae8e4 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:08.944215 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-c87ac9cd-5f17-4ed1-8cbe-6782cbb3b0ed None None] MFA Rules not processed for user `6da90312772a4cd6976146c834b993fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:08.945515 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-c87ac9cd-5f17-4ed1-8cbe-6782cbb3b0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:08.977521 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6372a0-3b35-4b67-a485-41d695bae8e4 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:08.985685 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b0bb4-38b8-4198-9f74-ed3f7d9be7a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:08.986842 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b0bb4-38b8-4198-9f74-ed3f7d9be7a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:08.993067 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b0bb4-38b8-4198-9f74-ed3f7d9be7a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:08.993616 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b0bb4-38b8-4198-9f74-ed3f7d9be7a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:09.008865 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b0bb4-38b8-4198-9f74-ed3f7d9be7a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:09.018297 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b0bb4-38b8-4198-9f74-ed3f7d9be7a4 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:09.029370 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:09.038586 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 800/1625] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:09.039969 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23c475-8e22-4bb5-b4ad-700331b62231 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:09.040322 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23c475-8e22-4bb5-b4ad-700331b62231 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:09.040623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23c475-8e22-4bb5-b4ad-700331b62231 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:09.042210 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 801/1626] 10.222.0.26 () {54 vars in 962 bytes} [Fri Aug 5 14:25:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:25:09.044765 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa23c475-8e22-4bb5-b4ad-700331b62231 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:09.078748 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa23c475-8e22-4bb5-b4ad-700331b62231 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:09.088958 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48fdcf9-a9a6-4167-8e00-10fc83c2677b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:09.089593 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fdcf9-a9a6-4167-8e00-10fc83c2677b admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:09.089875 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fdcf9-a9a6-4167-8e00-10fc83c2677b admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:09.090151 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fdcf9-a9a6-4167-8e00-10fc83c2677b admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:09.090849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48fdcf9-a9a6-4167-8e00-10fc83c2677b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:09.097518 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48fdcf9-a9a6-4167-8e00-10fc83c2677b admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:09.119032 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 826/1627] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:08 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:09.120388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 802/1628] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:09.120681 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ac9cd-5f17-4ed1-8cbe-6782cbb3b0ed None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:09.123991 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce16cdc-0de5-429c-b165-8fbd178398e4 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:09.126415 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce16cdc-0de5-429c-b165-8fbd178398e4 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:09.126714 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce16cdc-0de5-429c-b165-8fbd178398e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:09.144907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ac9cd-5f17-4ed1-8cbe-6782cbb3b0ed None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:09.160385 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ddf45-77f9-4d29-bb65-a09f0fbc6ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:09.161507 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ddf45-77f9-4d29-bb65-a09f0fbc6ada admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:09.161996 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ddf45-77f9-4d29-bb65-a09f0fbc6ada admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:09.162436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ddf45-77f9-4d29-bb65-a09f0fbc6ada admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:09.165620 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ddf45-77f9-4d29-bb65-a09f0fbc6ada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:09.168353 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ddf45-77f9-4d29-bb65-a09f0fbc6ada admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:09.197888 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 827/1629] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:09.197888 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ddf45-77f9-4d29-bb65-a09f0fbc6ada admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:09.214294 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-6ce16cdc-0de5-429c-b165-8fbd178398e4 None None] MFA Rules not processed for user `5ec4940c99fc4fa8a5bc7cdabc4558fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:09.215558 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-6ce16cdc-0de5-429c-b165-8fbd178398e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:09.218710 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ddf45-77f9-4d29-bb65-a09f0fbc6ada admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:09.231612 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ae81d-5f1a-4e4f-bd13-a011de9d4c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:09.235344 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ae81d-5f1a-4e4f-bd13-a011de9d4c0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:09.235344 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ae81d-5f1a-4e4f-bd13-a011de9d4c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:09.235344 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ae81d-5f1a-4e4f-bd13-a011de9d4c0b admin admin] PATH_INFO: `/v3/projects/b95ff5b46ca14d2c937997b7f37e03b8/users/1a8bf0e6f7c94b0d962e4373838ada44/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:09.275486 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87ae81d-5f1a-4e4f-bd13-a011de9d4c0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b95ff5b46ca14d2c937997b7f37e03b8, user_id=1a8bf0e6f7c94b0d962e4373838ada44, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:09.295754 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87ae81d-5f1a-4e4f-bd13-a011de9d4c0b admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:09.333576 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 803/1630] 10.222.0.26 () {58 vars in 1059 bytes} [Fri Aug 5 14:25:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:09.339112 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce16cdc-0de5-429c-b165-8fbd178398e4 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:09.400962 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 828/1631] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:25:09 2022] PUT /identity/v3/projects/b95ff5b46ca14d2c937997b7f37e03b8/users/1a8bf0e6f7c94b0d962e4373838ada44/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:09.402006 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ae81d-5f1a-4e4f-bd13-a011de9d4c0b admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:09.424668 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce16cdc-0de5-429c-b165-8fbd178398e4 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:09.504615 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03382f12-5d23-4a8e-8fe1-7649220ad5a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:09.505318 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03382f12-5d23-4a8e-8fe1-7649220ad5a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:09.506192 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03382f12-5d23-4a8e-8fe1-7649220ad5a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:09.506791 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03382f12-5d23-4a8e-8fe1-7649220ad5a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:09.529339 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:09.569091 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ae81d-5f1a-4e4f-bd13-a011de9d4c0b admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:09.580155 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606db64f-e76b-4b90-ada4-d231339d756c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:09.580913 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606db64f-e76b-4b90-ada4-d231339d756c admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:09.581191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606db64f-e76b-4b90-ada4-d231339d756c admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:09.581464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606db64f-e76b-4b90-ada4-d231339d756c admin admin] PATH_INFO: `/v3/projects/bee867c79b4849958b17ca3ba6995d90/users/b66ef9cc7044409d9c1dd51807f7dcd7/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:09.625496 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606db64f-e76b-4b90-ada4-d231339d756c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bee867c79b4849958b17ca3ba6995d90, user_id=b66ef9cc7044409d9c1dd51807f7dcd7, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:09.632470 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606db64f-e76b-4b90-ada4-d231339d756c admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:09.652014 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03382f12-5d23-4a8e-8fe1-7649220ad5a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:09.654123 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03382f12-5d23-4a8e-8fe1-7649220ad5a5 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:09.674533 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 804/1632] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:09 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:09.703163 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03382f12-5d23-4a8e-8fe1-7649220ad5a5 service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:09.726934 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:09.800464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 829/1633] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:25:09 2022] PUT /identity/v3/projects/bee867c79b4849958b17ca3ba6995d90/users/b66ef9cc7044409d9c1dd51807f7dcd7/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:09.800464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606db64f-e76b-4b90-ada4-d231339d756c admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:09.882417 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03382f12-5d23-4a8e-8fe1-7649220ad5a5 service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:09.937942 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606db64f-e76b-4b90-ada4-d231339d756c admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:09.951331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68d2983-7561-4b4f-a14e-89670235ec44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:09.951984 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d2983-7561-4b4f-a14e-89670235ec44 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:09.952245 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d2983-7561-4b4f-a14e-89670235ec44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:09.952553 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d2983-7561-4b4f-a14e-89670235ec44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:09.953261 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68d2983-7561-4b4f-a14e-89670235ec44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:09.954891 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68d2983-7561-4b4f-a14e-89670235ec44 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:09.955125 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae445655-15c8-41f5-a94c-acdde3a4507e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:09.955826 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae445655-15c8-41f5-a94c-acdde3a4507e service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:09.956108 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae445655-15c8-41f5-a94c-acdde3a4507e service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:09.956384 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae445655-15c8-41f5-a94c-acdde3a4507e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:09.978476 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 830/1634] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:09.980587 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68d2983-7561-4b4f-a14e-89670235ec44 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:10.004753 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68d2983-7561-4b4f-a14e-89670235ec44 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:10.014794 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25166360-a2df-4196-97ca-f1aefb792ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:10.015919 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25166360-a2df-4196-97ca-f1aefb792ef6 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:10.015919 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25166360-a2df-4196-97ca-f1aefb792ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:10.016165 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25166360-a2df-4196-97ca-f1aefb792ef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:10.016741 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25166360-a2df-4196-97ca-f1aefb792ef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:10.019583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25166360-a2df-4196-97ca-f1aefb792ef6 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:10.044502 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 831/1635] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:10.045435 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25166360-a2df-4196-97ca-f1aefb792ef6 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:10.098440 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae445655-15c8-41f5-a94c-acdde3a4507e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:10.101157 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae445655-15c8-41f5-a94c-acdde3a4507e service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:10.125357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 805/1636] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:10.126301 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae445655-15c8-41f5-a94c-acdde3a4507e service nova] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:10.139448 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae445655-15c8-41f5-a94c-acdde3a4507e service nova] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:10.144963 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ec4df-845a-42f8-a889-9c7d6579f15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:10.145637 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ec4df-845a-42f8-a889-9c7d6579f15a admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:10.145917 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ec4df-845a-42f8-a889-9c7d6579f15a admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:10.146251 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ec4df-845a-42f8-a889-9c7d6579f15a admin admin] PATH_INFO: `/v3/projects/b95ff5b46ca14d2c937997b7f37e03b8/users/1a8bf0e6f7c94b0d962e4373838ada44/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:10.149078 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651ec4df-845a-42f8-a889-9c7d6579f15a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b95ff5b46ca14d2c937997b7f37e03b8, user_id=1a8bf0e6f7c94b0d962e4373838ada44, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:10.160368 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651ec4df-845a-42f8-a889-9c7d6579f15a admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:10.189635 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25166360-a2df-4196-97ca-f1aefb792ef6 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:10.191721 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 806/1637] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:25:10 2022] PUT /identity/v3/projects/b95ff5b46ca14d2c937997b7f37e03b8/users/1a8bf0e6f7c94b0d962e4373838ada44/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:10.203171 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ec4df-845a-42f8-a889-9c7d6579f15a admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:10.210702 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:10.285187 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ec4df-845a-42f8-a889-9c7d6579f15a admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:10.291634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5449ef-75c6-4876-8739-798d985ab7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:10.292511 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5449ef-75c6-4876-8739-798d985ab7d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:10.292774 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5449ef-75c6-4876-8739-798d985ab7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:10.293367 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5449ef-75c6-4876-8739-798d985ab7d6 admin admin] PATH_INFO: `/v3/projects/bee867c79b4849958b17ca3ba6995d90/users/b66ef9cc7044409d9c1dd51807f7dcd7/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:10.296160 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5449ef-75c6-4876-8739-798d985ab7d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bee867c79b4849958b17ca3ba6995d90, user_id=b66ef9cc7044409d9c1dd51807f7dcd7, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:10.296350 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056547b1-18ef-46b8-9d74-490ce78ae449 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:10.298253 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056547b1-18ef-46b8-9d74-490ce78ae449 service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:10.298960 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5449ef-75c6-4876-8739-798d985ab7d6 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:10.299806 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056547b1-18ef-46b8-9d74-490ce78ae449 service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:10.300862 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056547b1-18ef-46b8-9d74-490ce78ae449 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:10.323558 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 807/1638] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:25:10 2022] PUT /identity/v3/projects/bee867c79b4849958b17ca3ba6995d90/users/b66ef9cc7044409d9c1dd51807f7dcd7/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:10.323558 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840897ca-08d4-4f2d-8cef-879569c63583 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:10.323558 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840897ca-08d4-4f2d-8cef-879569c63583 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:10.323558 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840897ca-08d4-4f2d-8cef-879569c63583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:10.356598 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-840897ca-08d4-4f2d-8cef-879569c63583 None None] MFA Rules not processed for user `1a8bf0e6f7c94b0d962e4373838ada44`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:10.357850 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-840897ca-08d4-4f2d-8cef-879569c63583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:10.395954 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:10.433755 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056547b1-18ef-46b8-9d74-490ce78ae449 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:10.437160 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056547b1-18ef-46b8-9d74-490ce78ae449 service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:10.452335 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 808/1639] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:10 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:10.453974 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3155c3f-0088-4008-aa13-5959a9c0b087 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:10.454267 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3155c3f-0088-4008-aa13-5959a9c0b087 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:10.454537 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3155c3f-0088-4008-aa13-5959a9c0b087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:10.470465 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 832/1640] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:10 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:10.477328 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056547b1-18ef-46b8-9d74-490ce78ae449 service cinder] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:10.489685 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-d3155c3f-0088-4008-aa13-5959a9c0b087 None None] MFA Rules not processed for user `b66ef9cc7044409d9c1dd51807f7dcd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:10.491844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d3155c3f-0088-4008-aa13-5959a9c0b087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:10.496978 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056547b1-18ef-46b8-9d74-490ce78ae449 service cinder] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:10.588543 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 809/1641] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:10 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:10.630249 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81d6a8-247b-46f9-8fdc-74fb419ec191 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:10.631039 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81d6a8-247b-46f9-8fdc-74fb419ec191 service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:10.633789 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81d6a8-247b-46f9-8fdc-74fb419ec191 service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:10.634307 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81d6a8-247b-46f9-8fdc-74fb419ec191 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:10.657730 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef81d6a8-247b-46f9-8fdc-74fb419ec191 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:10.663847 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef81d6a8-247b-46f9-8fdc-74fb419ec191 service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:10.686493 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 833/1642] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:10 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:10.704065 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995cb00-51ce-4220-b1e7-c2a36e5c79f4 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:10.705011 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995cb00-51ce-4220-b1e7-c2a36e5c79f4 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:10.705316 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995cb00-51ce-4220-b1e7-c2a36e5c79f4 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:10.707386 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 810/1643] 10.222.0.26 () {54 vars in 912 bytes} [Fri Aug 5 14:25:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:25:10.714488 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3770e23-e41b-4533-987a-4c9b56d22dd1 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:10.714783 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3770e23-e41b-4533-987a-4c9b56d22dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:10.715436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3770e23-e41b-4533-987a-4c9b56d22dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:10.718419 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:10.755536 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-f3770e23-e41b-4533-987a-4c9b56d22dd1 None None] MFA Rules not processed for user `5ec4940c99fc4fa8a5bc7cdabc4558fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:10.761781 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-f3770e23-e41b-4533-987a-4c9b56d22dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:11.070715 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f995cb00-51ce-4220-b1e7-c2a36e5c79f4 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:11.070715 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f995cb00-51ce-4220-b1e7-c2a36e5c79f4 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:11.070715 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c10b7-5322-4951-856c-6ff9942796a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:11.070715 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c10b7-5322-4951-856c-6ff9942796a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:11.071814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c10b7-5322-4951-856c-6ff9942796a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:11.071814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c10b7-5322-4951-856c-6ff9942796a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:11.071814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c10b7-5322-4951-856c-6ff9942796a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:11.071814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c10b7-5322-4951-856c-6ff9942796a8 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:11.071814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 811/1644] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:10 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:11.072261 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 834/1645] 10.222.0.26 () {58 vars in 1009 bytes} [Fri Aug 5 14:25:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:11.351983 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c10b7-5322-4951-856c-6ff9942796a8 service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:11.363305 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c10b7-5322-4951-856c-6ff9942796a8 service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:11.368152 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff62c02-10f2-4df8-aa15-44fe52430f06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:11.369207 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff62c02-10f2-4df8-aa15-44fe52430f06 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:11.369650 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff62c02-10f2-4df8-aa15-44fe52430f06 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:11.370060 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff62c02-10f2-4df8-aa15-44fe52430f06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:11.401888 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff62c02-10f2-4df8-aa15-44fe52430f06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:11.407668 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff62c02-10f2-4df8-aa15-44fe52430f06 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:11.428815 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 812/1646] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 14:25:13.763757 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e998d-38db-46c7-97af-9c2928e9244c None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:13.764264 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e998d-38db-46c7-97af-9c2928e9244c None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:13.764361 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e998d-38db-46c7-97af-9c2928e9244c None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:13.765964 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 835/1647] 10.222.0.26 () {54 vars in 962 bytes} [Fri Aug 5 14:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:25:13.777786 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ef8d3-c1c3-4256-b6cf-a42fcffb1335 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:13.778318 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ef8d3-c1c3-4256-b6cf-a42fcffb1335 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:13.778599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ef8d3-c1c3-4256-b6cf-a42fcffb1335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:13.847098 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-3b7ef8d3-c1c3-4256-b6cf-a42fcffb1335 None None] MFA Rules not processed for user `5ec4940c99fc4fa8a5bc7cdabc4558fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:13.856372 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-3b7ef8d3-c1c3-4256-b6cf-a42fcffb1335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:13.897551 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 813/1648] 10.222.0.26 () {58 vars in 1059 bytes} [Fri Aug 5 14:25:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:25:13.903992 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7c029-e63c-4f99-aec6-5c2ceb4e36b1 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:13.904604 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7c029-e63c-4f99-aec6-5c2ceb4e36b1 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:13.905038 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7c029-e63c-4f99-aec6-5c2ceb4e36b1 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:13.907082 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 836/1649] 10.222.0.26 () {54 vars in 912 bytes} [Fri Aug 5 14:25:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:25:13.913597 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503df488-6e21-4f93-8acd-0887bd1b846e None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:13.914026 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503df488-6e21-4f93-8acd-0887bd1b846e None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:13.914459 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503df488-6e21-4f93-8acd-0887bd1b846e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:14.016303 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-503df488-6e21-4f93-8acd-0887bd1b846e None None] MFA Rules not processed for user `5ec4940c99fc4fa8a5bc7cdabc4558fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:14.021777 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-503df488-6e21-4f93-8acd-0887bd1b846e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:14.050864 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 814/1650] 10.222.0.26 () {58 vars in 1009 bytes} [Fri Aug 5 14:25:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:14.140344 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7c029-e63c-4f99-aec6-5c2ceb4e36b1 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:14.220036 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503df488-6e21-4f93-8acd-0887bd1b846e None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:14.289195 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7c029-e63c-4f99-aec6-5c2ceb4e36b1 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:14.313203 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995d152-bbd8-4237-b1a2-243dfbde49b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:14.313906 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995d152-bbd8-4237-b1a2-243dfbde49b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:14.314348 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995d152-bbd8-4237-b1a2-243dfbde49b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:14.314784 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995d152-bbd8-4237-b1a2-243dfbde49b9 admin admin] PATH_INFO: `/v3/users/2870756169cd4516bf733bc2f257186c` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:14.317388 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0995d152-bbd8-4237-b1a2-243dfbde49b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2870756169cd4516bf733bc2f257186c)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:14.327939 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0995d152-bbd8-4237-b1a2-243dfbde49b9 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:14.335269 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:14.359674 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503df488-6e21-4f93-8acd-0887bd1b846e None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:14.370825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93300936-a14f-4856-a31f-99e64f80cdd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:14.371802 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93300936-a14f-4856-a31f-99e64f80cdd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:14.372262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93300936-a14f-4856-a31f-99e64f80cdd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:14.372773 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93300936-a14f-4856-a31f-99e64f80cdd5 admin admin] PATH_INFO: `/v3/users/c6da0409312749478378d65d478265ba` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:14.389506 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93300936-a14f-4856-a31f-99e64f80cdd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6da0409312749478378d65d478265ba)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:14.389506 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: WARNING py.warnings [None req-93300936-a14f-4856-a31f-99e64f80cdd5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 14:25:14.389506 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: warnings.warn(msg) Aug 05 14:25:14.389506 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: Aug 05 14:25:14.389506 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93300936-a14f-4856-a31f-99e64f80cdd5 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:14.688625 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-0995d152-bbd8-4237-b1a2-243dfbde49b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2870756169cd4516bf733bc2f257186c'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:14.710726 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-0995d152-bbd8-4237-b1a2-243dfbde49b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2870756169cd4516bf733bc2f257186c'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:14.712171 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-0995d152-bbd8-4237-b1a2-243dfbde49b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2870756169cd4516bf733bc2f257186c'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:14.713241 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-0995d152-bbd8-4237-b1a2-243dfbde49b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2870756169cd4516bf733bc2f257186c'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:14.719211 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-93300936-a14f-4856-a31f-99e64f80cdd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6da0409312749478378d65d478265ba'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:14.737028 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-0995d152-bbd8-4237-b1a2-243dfbde49b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2870756169cd4516bf733bc2f257186c'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:14.748175 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-93300936-a14f-4856-a31f-99e64f80cdd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6da0409312749478378d65d478265ba'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:14.867351 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 837/1651] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:25:14 2022] DELETE /identity/v3/users/2870756169cd4516bf733bc2f257186c => generated 0 bytes in 727 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:14.868920 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995d152-bbd8-4237-b1a2-243dfbde49b9 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:14.877710 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-0995d152-bbd8-4237-b1a2-243dfbde49b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:14.983177 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-93300936-a14f-4856-a31f-99e64f80cdd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6da0409312749478378d65d478265ba'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:14.983885 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-93300936-a14f-4856-a31f-99e64f80cdd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6da0409312749478378d65d478265ba'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:15.007789 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-93300936-a14f-4856-a31f-99e64f80cdd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6da0409312749478378d65d478265ba'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:15.036265 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 815/1652] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:25:14 2022] DELETE /identity/v3/users/c6da0409312749478378d65d478265ba => generated 0 bytes in 819 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:15.043478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b8f00-616a-4d83-9f78-01e84fa92d0f None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:15.045263 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b8f00-616a-4d83-9f78-01e84fa92d0f None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:15.045555 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b8f00-616a-4d83-9f78-01e84fa92d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:15.094633 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995d152-bbd8-4237-b1a2-243dfbde49b9 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:15.248286 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-e66b8f00-616a-4d83-9f78-01e84fa92d0f None None] MFA Rules not processed for user `b9d62fe4e6de4308a3a51be3e900625f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:15.249588 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e66b8f00-616a-4d83-9f78-01e84fa92d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:15.375805 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:15.390160 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54c945-525a-4f68-8ca0-9ff695639228 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:15.396942 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54c945-525a-4f68-8ca0-9ff695639228 service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:15.396942 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54c945-525a-4f68-8ca0-9ff695639228 service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:15.396942 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54c945-525a-4f68-8ca0-9ff695639228 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:15.403066 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-8a54c945-525a-4f68-8ca0-9ff695639228 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:15.554092 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 816/1653] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:15 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 517 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:15.556006 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde2dc9-af36-4380-b7a9-b69b056244c5 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:15.556296 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde2dc9-af36-4380-b7a9-b69b056244c5 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:15.557823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde2dc9-af36-4380-b7a9-b69b056244c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:15.637771 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:15.640367 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-cbde2dc9-af36-4380-b7a9-b69b056244c5 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:15.641398 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-cbde2dc9-af36-4380-b7a9-b69b056244c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:15.726608 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a54c945-525a-4f68-8ca0-9ff695639228 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:15.728205 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a54c945-525a-4f68-8ca0-9ff695639228 service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:15.777171 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 838/1654] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 909 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:15.780089 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ec34b-fb2f-420a-b3db-316dfff7ab24 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:15.780376 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ec34b-fb2f-420a-b3db-316dfff7ab24 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:15.780680 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ec34b-fb2f-420a-b3db-316dfff7ab24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:15.831841 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-8f8ec34b-fb2f-420a-b3db-316dfff7ab24 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:15.833118 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-8f8ec34b-fb2f-420a-b3db-316dfff7ab24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:15.865759 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 817/1655] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:15.868162 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62951d-5e13-44cc-bcea-e5f657a973d2 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:15.874283 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62951d-5e13-44cc-bcea-e5f657a973d2 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:15.877670 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62951d-5e13-44cc-bcea-e5f657a973d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:15.978994 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-4d62951d-5e13-44cc-bcea-e5f657a973d2 None None] MFA Rules not processed for user `b9d62fe4e6de4308a3a51be3e900625f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:15.980519 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-4d62951d-5e13-44cc-bcea-e5f657a973d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:16.011554 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 818/1656] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:15 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:16.013448 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62951d-5e13-44cc-bcea-e5f657a973d2 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:16.018385 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-4d62951d-5e13-44cc-bcea-e5f657a973d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:16.079936 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 839/1657] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:16.082462 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ec34b-fb2f-420a-b3db-316dfff7ab24 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:16.084916 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-8f8ec34b-fb2f-420a-b3db-316dfff7ab24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:16.123628 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ec34b-fb2f-420a-b3db-316dfff7ab24 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:16.138694 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f3e73-9e88-438e-8bf4-b2875d3438fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:16.140643 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f3e73-9e88-438e-8bf4-b2875d3438fb service nova] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:16.141116 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f3e73-9e88-438e-8bf4-b2875d3438fb service nova] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:16.142423 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f3e73-9e88-438e-8bf4-b2875d3438fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:16.158062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62951d-5e13-44cc-bcea-e5f657a973d2 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:16.160159 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f3e73-9e88-438e-8bf4-b2875d3438fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:16.162613 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f3e73-9e88-438e-8bf4-b2875d3438fb service nova] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:16.169844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d09dda6-b53a-4699-a967-8b29bee9554b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:16.170848 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09dda6-b53a-4699-a967-8b29bee9554b service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:16.171311 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09dda6-b53a-4699-a967-8b29bee9554b service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:16.171761 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09dda6-b53a-4699-a967-8b29bee9554b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:16.184827 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 840/1658] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:16 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:16.188812 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f3e73-9e88-438e-8bf4-b2875d3438fb service nova] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:16.193575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-b90f3e73-9e88-438e-8bf4-b2875d3438fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:16.197784 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d09dda6-b53a-4699-a967-8b29bee9554b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:16.197784 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d09dda6-b53a-4699-a967-8b29bee9554b service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:16.219585 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f3e73-9e88-438e-8bf4-b2875d3438fb service nova] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:16.224792 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 819/1659] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:16.238779 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e5ee1-84bf-490c-addf-51c0d9cbaf92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:16.239425 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e5ee1-84bf-490c-addf-51c0d9cbaf92 service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:16.240323 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e5ee1-84bf-490c-addf-51c0d9cbaf92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:16.240715 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e5ee1-84bf-490c-addf-51c0d9cbaf92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:16.261698 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e5ee1-84bf-490c-addf-51c0d9cbaf92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:16.263166 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e5ee1-84bf-490c-addf-51c0d9cbaf92 service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:16.287554 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 841/1660] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:17.217924 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82c59d-1205-4b49-b3a4-9833690e638f None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:17.219325 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82c59d-1205-4b49-b3a4-9833690e638f None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:17.219850 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82c59d-1205-4b49-b3a4-9833690e638f None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:17.222078 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 820/1661] 10.222.0.26 () {54 vars in 912 bytes} [Fri Aug 5 14:25:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:25:17.232876 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c469fc-3d62-4005-8b05-790c01df2bb3 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:17.233831 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c469fc-3d62-4005-8b05-790c01df2bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:17.234383 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c469fc-3d62-4005-8b05-790c01df2bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:17.291317 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da82c59d-1205-4b49-b3a4-9833690e638f None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:17.293393 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-da82c59d-1205-4b49-b3a4-9833690e638f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:17.325492 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da82c59d-1205-4b49-b3a4-9833690e638f None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:17.334235 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-49c469fc-3d62-4005-8b05-790c01df2bb3 None None] MFA Rules not processed for user `695e37560f8f495e987d1bd17acc2680`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:17.334235 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-49c469fc-3d62-4005-8b05-790c01df2bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:17.338864 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a03242-f10a-4bcd-90ca-77bd33e19dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:17.339842 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a03242-f10a-4bcd-90ca-77bd33e19dc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:17.340274 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a03242-f10a-4bcd-90ca-77bd33e19dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:17.340938 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a03242-f10a-4bcd-90ca-77bd33e19dc9 admin admin] PATH_INFO: `/v3/projects/dbe84e660595496fbff4fb67c998c5f3` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:17.342967 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a03242-f10a-4bcd-90ca-77bd33e19dc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbe84e660595496fbff4fb67c998c5f3)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:17.347947 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a03242-f10a-4bcd-90ca-77bd33e19dc9 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:17.353162 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 842/1662] 10.222.0.26 () {58 vars in 1009 bytes} [Fri Aug 5 14:25:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:17.423563 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c469fc-3d62-4005-8b05-790c01df2bb3 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:17.435624 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c469fc-3d62-4005-8b05-790c01df2bb3 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:17.442381 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176cfa34-6e1a-453b-af16-06152c03200d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:17.443226 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176cfa34-6e1a-453b-af16-06152c03200d service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:17.443661 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176cfa34-6e1a-453b-af16-06152c03200d service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:17.444120 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176cfa34-6e1a-453b-af16-06152c03200d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:17.466238 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176cfa34-6e1a-453b-af16-06152c03200d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:17.469787 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176cfa34-6e1a-453b-af16-06152c03200d service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:17.506041 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 843/1663] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:17.766911 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-42a03242-f10a-4bcd-90ca-77bd33e19dc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbe84e660595496fbff4fb67c998c5f3', 'request_id': 'req-42a03242-f10a-4bcd-90ca-77bd33e19dc9'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:17.793658 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-42a03242-f10a-4bcd-90ca-77bd33e19dc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbe84e660595496fbff4fb67c998c5f3', 'request_id': 'req-42a03242-f10a-4bcd-90ca-77bd33e19dc9'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:17.803093 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:17.969058 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-42a03242-f10a-4bcd-90ca-77bd33e19dc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbe84e660595496fbff4fb67c998c5f3', 'request_id': 'req-42a03242-f10a-4bcd-90ca-77bd33e19dc9'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:17.986882 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-42a03242-f10a-4bcd-90ca-77bd33e19dc9 admin admin] The token cache is being invalidate because project dbe84e660595496fbff4fb67c998c5f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:25:17.988813 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-42a03242-f10a-4bcd-90ca-77bd33e19dc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:17.990058 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-42a03242-f10a-4bcd-90ca-77bd33e19dc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:17.998718 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 821/1664] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:25:17 2022] DELETE /identity/v3/projects/dbe84e660595496fbff4fb67c998c5f3 => generated 0 bytes in 706 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:18.000758 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176cfa34-6e1a-453b-af16-06152c03200d service neutron] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:18.023024 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176cfa34-6e1a-453b-af16-06152c03200d service neutron] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:18.219479 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e7b1a-e110-4a04-a0f8-9b31b641aa47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:18.220294 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e7b1a-e110-4a04-a0f8-9b31b641aa47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:18.221612 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e7b1a-e110-4a04-a0f8-9b31b641aa47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:18.221998 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e7b1a-e110-4a04-a0f8-9b31b641aa47 admin admin] PATH_INFO: `/v3/users/1a8bf0e6f7c94b0d962e4373838ada44` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:18.224941 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2e7b1a-e110-4a04-a0f8-9b31b641aa47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a8bf0e6f7c94b0d962e4373838ada44)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:18.227035 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2e7b1a-e110-4a04-a0f8-9b31b641aa47 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:18.264595 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:18.549213 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-0b2e7b1a-e110-4a04-a0f8-9b31b641aa47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a8bf0e6f7c94b0d962e4373838ada44'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:18.561673 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-0b2e7b1a-e110-4a04-a0f8-9b31b641aa47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a8bf0e6f7c94b0d962e4373838ada44'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:18.562740 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-0b2e7b1a-e110-4a04-a0f8-9b31b641aa47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a8bf0e6f7c94b0d962e4373838ada44'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:18.564051 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-0b2e7b1a-e110-4a04-a0f8-9b31b641aa47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a8bf0e6f7c94b0d962e4373838ada44'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:18.583950 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-0b2e7b1a-e110-4a04-a0f8-9b31b641aa47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a8bf0e6f7c94b0d962e4373838ada44'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:18.626393 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a03242-f10a-4bcd-90ca-77bd33e19dc9 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:18.631276 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-42a03242-f10a-4bcd-90ca-77bd33e19dc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:18.657164 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a03242-f10a-4bcd-90ca-77bd33e19dc9 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:18.664083 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3155ff18-aace-4f59-afb9-72cc430858f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:18.665517 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155ff18-aace-4f59-afb9-72cc430858f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:18.665899 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155ff18-aace-4f59-afb9-72cc430858f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:18.666319 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155ff18-aace-4f59-afb9-72cc430858f8 admin admin] PATH_INFO: `/v3/projects/8ba19526865c4d699b2521c3fd5b2847` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:18.667780 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3155ff18-aace-4f59-afb9-72cc430858f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ba19526865c4d699b2521c3fd5b2847)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:18.670120 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3155ff18-aace-4f59-afb9-72cc430858f8 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:18.750197 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 844/1665] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:25:17 2022] DELETE /identity/v3/users/1a8bf0e6f7c94b0d962e4373838ada44 => generated 0 bytes in 750 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:18.850608 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-3155ff18-aace-4f59-afb9-72cc430858f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ba19526865c4d699b2521c3fd5b2847', 'request_id': 'req-3155ff18-aace-4f59-afb9-72cc430858f8'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:18.866593 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-3155ff18-aace-4f59-afb9-72cc430858f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ba19526865c4d699b2521c3fd5b2847', 'request_id': 'req-3155ff18-aace-4f59-afb9-72cc430858f8'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:18.991897 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-3155ff18-aace-4f59-afb9-72cc430858f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ba19526865c4d699b2521c3fd5b2847', 'request_id': 'req-3155ff18-aace-4f59-afb9-72cc430858f8'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:19.006767 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-3155ff18-aace-4f59-afb9-72cc430858f8 admin admin] The token cache is being invalidate because project 8ba19526865c4d699b2521c3fd5b2847 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:25:19.007065 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-3155ff18-aace-4f59-afb9-72cc430858f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:19.009859 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-3155ff18-aace-4f59-afb9-72cc430858f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:19.013007 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 822/1666] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:25:18 2022] DELETE /identity/v3/projects/8ba19526865c4d699b2521c3fd5b2847 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:25:19.017113 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e7b1a-e110-4a04-a0f8-9b31b641aa47 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:19.019516 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-0b2e7b1a-e110-4a04-a0f8-9b31b641aa47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:19.086669 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3155ff18-aace-4f59-afb9-72cc430858f8 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:19.088647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-3155ff18-aace-4f59-afb9-72cc430858f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:19.209481 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e7b1a-e110-4a04-a0f8-9b31b641aa47 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:19.277312 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3155ff18-aace-4f59-afb9-72cc430858f8 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:19.278422 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f40eb2-d62a-4ebe-9b1a-ddf3e16b9ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:19.279706 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f40eb2-d62a-4ebe-9b1a-ddf3e16b9ec3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:19.280302 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f40eb2-d62a-4ebe-9b1a-ddf3e16b9ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:19.281135 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f40eb2-d62a-4ebe-9b1a-ddf3e16b9ec3 admin admin] PATH_INFO: `/v3/users/b66ef9cc7044409d9c1dd51807f7dcd7` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:19.282927 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f40eb2-d62a-4ebe-9b1a-ddf3e16b9ec3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b66ef9cc7044409d9c1dd51807f7dcd7)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:19.283892 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa6602-d077-4ede-8efc-03e0cb5f3622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:19.284638 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa6602-d077-4ede-8efc-03e0cb5f3622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:19.284915 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa6602-d077-4ede-8efc-03e0cb5f3622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:19.285174 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa6602-d077-4ede-8efc-03e0cb5f3622 admin admin] PATH_INFO: `/v3/users/a9212230bba44398af36ffdee8a0d62d` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:19.286789 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f40eb2-d62a-4ebe-9b1a-ddf3e16b9ec3 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:19.287538 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaa6602-d077-4ede-8efc-03e0cb5f3622 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9212230bba44398af36ffdee8a0d62d)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:19.290107 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaa6602-d077-4ede-8efc-03e0cb5f3622 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:19.565806 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ddaa6602-d077-4ede-8efc-03e0cb5f3622 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9212230bba44398af36ffdee8a0d62d'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:19.570707 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-a6f40eb2-d62a-4ebe-9b1a-ddf3e16b9ec3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b66ef9cc7044409d9c1dd51807f7dcd7'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:19.592331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-a6f40eb2-d62a-4ebe-9b1a-ddf3e16b9ec3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b66ef9cc7044409d9c1dd51807f7dcd7'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:19.593159 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-a6f40eb2-d62a-4ebe-9b1a-ddf3e16b9ec3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b66ef9cc7044409d9c1dd51807f7dcd7'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:19.596674 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-a6f40eb2-d62a-4ebe-9b1a-ddf3e16b9ec3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b66ef9cc7044409d9c1dd51807f7dcd7'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:19.603466 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ddaa6602-d077-4ede-8efc-03e0cb5f3622 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9212230bba44398af36ffdee8a0d62d'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:19.635110 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-a6f40eb2-d62a-4ebe-9b1a-ddf3e16b9ec3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b66ef9cc7044409d9c1dd51807f7dcd7'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:19.745837 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ddaa6602-d077-4ede-8efc-03e0cb5f3622 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9212230bba44398af36ffdee8a0d62d'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:19.747157 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ddaa6602-d077-4ede-8efc-03e0cb5f3622 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9212230bba44398af36ffdee8a0d62d'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:19.762801 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ddaa6602-d077-4ede-8efc-03e0cb5f3622 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9212230bba44398af36ffdee8a0d62d'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:19.776978 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 823/1667] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:25:19 2022] DELETE /identity/v3/users/a9212230bba44398af36ffdee8a0d62d => generated 0 bytes in 688 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:19.780031 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d39a43-5a9f-47b0-97e4-f950a3b86280 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:19.780538 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d39a43-5a9f-47b0-97e4-f950a3b86280 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:19.781005 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d39a43-5a9f-47b0-97e4-f950a3b86280 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:19.783964 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 824/1668] 10.222.0.26 () {54 vars in 961 bytes} [Fri Aug 5 14:25:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:25:19.785359 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8396f-3ed8-471f-87d1-ca3569525991 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:19.791008 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8396f-3ed8-471f-87d1-ca3569525991 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:19.791561 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8396f-3ed8-471f-87d1-ca3569525991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:19.837930 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-90e8396f-3ed8-471f-87d1-ca3569525991 None None] MFA Rules not processed for user `b9d62fe4e6de4308a3a51be3e900625f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:19.839499 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-90e8396f-3ed8-471f-87d1-ca3569525991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:19.854250 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 845/1669] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:25:19 2022] DELETE /identity/v3/users/b66ef9cc7044409d9c1dd51807f7dcd7 => generated 0 bytes in 838 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:19.856029 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd1f85-2e68-4bae-b1d2-cc3df5764c90 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:19.858321 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd1f85-2e68-4bae-b1d2-cc3df5764c90 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:19.859149 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd1f85-2e68-4bae-b1d2-cc3df5764c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:19.869606 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:19.900163 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-82cd1f85-2e68-4bae-b1d2-cc3df5764c90 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:19.902075 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-82cd1f85-2e68-4bae-b1d2-cc3df5764c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:20.038829 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 846/1670] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:20.039772 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395fd8ab-815e-4373-8901-d5b7309829aa None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:20.040623 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395fd8ab-815e-4373-8901-d5b7309829aa None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:20.044146 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395fd8ab-815e-4373-8901-d5b7309829aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:20.051422 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 825/1671] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:19 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:20.053113 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e8396f-3ed8-471f-87d1-ca3569525991 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:20.058721 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-90e8396f-3ed8-471f-87d1-ca3569525991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:20.090390 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-395fd8ab-815e-4373-8901-d5b7309829aa None None] MFA Rules not processed for user `fff761f6f81a46098f94bd2ec2be9c1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:20.093127 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-395fd8ab-815e-4373-8901-d5b7309829aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:20.250316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 847/1672] 10.222.0.26 () {58 vars in 1058 bytes} [Fri Aug 5 14:25:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:20.259937 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395fd8ab-815e-4373-8901-d5b7309829aa None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:20.279092 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395fd8ab-815e-4373-8901-d5b7309829aa None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:20.288668 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:20.291137 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c67884-9ba7-4c0f-a5da-d77eb1f7511d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff761f6f81a46098f94bd2ec2be9c1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:20.291785 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c67884-9ba7-4c0f-a5da-d77eb1f7511d service placement] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:20.292250 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c67884-9ba7-4c0f-a5da-d77eb1f7511d service placement] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:20.292739 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c67884-9ba7-4c0f-a5da-d77eb1f7511d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:20.304152 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-48c67884-9ba7-4c0f-a5da-d77eb1f7511d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:20.312114 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c67884-9ba7-4c0f-a5da-d77eb1f7511d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:20.314174 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c67884-9ba7-4c0f-a5da-d77eb1f7511d service placement] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:20.332807 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 848/1673] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 14:25:20.337808 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c67884-9ba7-4c0f-a5da-d77eb1f7511d service placement] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:20.339324 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-48c67884-9ba7-4c0f-a5da-d77eb1f7511d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:20.343258 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e8396f-3ed8-471f-87d1-ca3569525991 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:20.365153 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0bcbe-d81b-4331-bb03-51bfe0cf7471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:20.367514 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0bcbe-d81b-4331-bb03-51bfe0cf7471 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:20.367514 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0bcbe-d81b-4331-bb03-51bfe0cf7471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:20.367514 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0bcbe-d81b-4331-bb03-51bfe0cf7471 admin admin] PATH_INFO: `/v3/projects/b95ff5b46ca14d2c937997b7f37e03b8` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:20.367728 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e0bcbe-d81b-4331-bb03-51bfe0cf7471 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b95ff5b46ca14d2c937997b7f37e03b8)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:20.375784 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e0bcbe-d81b-4331-bb03-51bfe0cf7471 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:20.499756 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c67884-9ba7-4c0f-a5da-d77eb1f7511d service placement] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:20.512914 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41303634-054f-4059-9e86-85027d83a70e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:20.513779 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41303634-054f-4059-9e86-85027d83a70e service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:20.513907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41303634-054f-4059-9e86-85027d83a70e service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:20.514101 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41303634-054f-4059-9e86-85027d83a70e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:20.538929 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41303634-054f-4059-9e86-85027d83a70e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:20.541839 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41303634-054f-4059-9e86-85027d83a70e service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:20.570186 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 849/1674] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:20.572240 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-72e0bcbe-d81b-4331-bb03-51bfe0cf7471 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b95ff5b46ca14d2c937997b7f37e03b8', 'request_id': 'req-72e0bcbe-d81b-4331-bb03-51bfe0cf7471'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:20.610341 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-72e0bcbe-d81b-4331-bb03-51bfe0cf7471 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b95ff5b46ca14d2c937997b7f37e03b8', 'request_id': 'req-72e0bcbe-d81b-4331-bb03-51bfe0cf7471'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:20.716830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-72e0bcbe-d81b-4331-bb03-51bfe0cf7471 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b95ff5b46ca14d2c937997b7f37e03b8', 'request_id': 'req-72e0bcbe-d81b-4331-bb03-51bfe0cf7471'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:20.726636 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41303634-054f-4059-9e86-85027d83a70e service neutron] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:20.729393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-41303634-054f-4059-9e86-85027d83a70e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:20.747084 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-72e0bcbe-d81b-4331-bb03-51bfe0cf7471 admin admin] The token cache is being invalidate because project b95ff5b46ca14d2c937997b7f37e03b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:25:20.750528 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41303634-054f-4059-9e86-85027d83a70e service neutron] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:20.751843 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-72e0bcbe-d81b-4331-bb03-51bfe0cf7471 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:20.753781 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-72e0bcbe-d81b-4331-bb03-51bfe0cf7471 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:20.761285 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 826/1675] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:25:20 2022] DELETE /identity/v3/projects/b95ff5b46ca14d2c937997b7f37e03b8 => generated 0 bytes in 707 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:20.764167 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa0e9e-adc3-44ef-88f6-0075356e1398 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:20.764479 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa0e9e-adc3-44ef-88f6-0075356e1398 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:20.764758 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa0e9e-adc3-44ef-88f6-0075356e1398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:20.836152 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-aaaa0e9e-adc3-44ef-88f6-0075356e1398 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:20.839808 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-aaaa0e9e-adc3-44ef-88f6-0075356e1398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:20.931064 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d54750-2e51-4d9b-af61-2396c5a14fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:20.932132 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d54750-2e51-4d9b-af61-2396c5a14fe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:20.932635 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d54750-2e51-4d9b-af61-2396c5a14fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:20.933175 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d54750-2e51-4d9b-af61-2396c5a14fe9 admin admin] PATH_INFO: `/v3/projects/bee867c79b4849958b17ca3ba6995d90` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:20.938838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d54750-2e51-4d9b-af61-2396c5a14fe9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bee867c79b4849958b17ca3ba6995d90)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:20.940064 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: WARNING py.warnings [None req-a0d54750-2e51-4d9b-af61-2396c5a14fe9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 14:25:20.940064 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: warnings.warn(msg) Aug 05 14:25:20.940064 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: Aug 05 14:25:20.941633 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d54750-2e51-4d9b-af61-2396c5a14fe9 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:20.996806 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 827/1676] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:20.998633 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339e6d2-6569-45cc-b097-5f282891e715 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:20.998905 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339e6d2-6569-45cc-b097-5f282891e715 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:20.999235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339e6d2-6569-45cc-b097-5f282891e715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:21.040115 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-a339e6d2-6569-45cc-b097-5f282891e715 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:21.041993 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a339e6d2-6569-45cc-b097-5f282891e715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:21.053410 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 828/1677] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:25:21.054568 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339e6d2-6569-45cc-b097-5f282891e715 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:21.065262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339e6d2-6569-45cc-b097-5f282891e715 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:21.073725 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba7dc1-cb4e-40f9-be51-bf9c4475d3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:21.074378 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba7dc1-cb4e-40f9-be51-bf9c4475d3a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:21.074658 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba7dc1-cb4e-40f9-be51-bf9c4475d3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:21.074929 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba7dc1-cb4e-40f9-be51-bf9c4475d3a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:21.090060 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba7dc1-cb4e-40f9-be51-bf9c4475d3a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:25:21.090624 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba7dc1-cb4e-40f9-be51-bf9c4475d3a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:21.093863 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba7dc1-cb4e-40f9-be51-bf9c4475d3a0 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:21.111347 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 829/1678] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:25:21.115980 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba7dc1-cb4e-40f9-be51-bf9c4475d3a0 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:21.131533 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba7dc1-cb4e-40f9-be51-bf9c4475d3a0 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:21.145683 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35de9bb-a6b5-4390-8a7e-3bee4f49a85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:21.146797 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35de9bb-a6b5-4390-8a7e-3bee4f49a85a admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:21.147262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35de9bb-a6b5-4390-8a7e-3bee4f49a85a admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:21.147725 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35de9bb-a6b5-4390-8a7e-3bee4f49a85a admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:21.148603 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35de9bb-a6b5-4390-8a7e-3bee4f49a85a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:21.150696 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35de9bb-a6b5-4390-8a7e-3bee4f49a85a admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:21.166280 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-a0d54750-2e51-4d9b-af61-2396c5a14fe9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bee867c79b4849958b17ca3ba6995d90', 'request_id': 'req-a0d54750-2e51-4d9b-af61-2396c5a14fe9'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:21.189128 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-a0d54750-2e51-4d9b-af61-2396c5a14fe9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bee867c79b4849958b17ca3ba6995d90', 'request_id': 'req-a0d54750-2e51-4d9b-af61-2396c5a14fe9'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:21.197087 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-a0d54750-2e51-4d9b-af61-2396c5a14fe9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bee867c79b4849958b17ca3ba6995d90', 'request_id': 'req-a0d54750-2e51-4d9b-af61-2396c5a14fe9'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:21.319059 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 830/1679] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:25:21 2022] POST /identity/v3/projects => generated 361 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:25:21.322991 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35de9bb-a6b5-4390-8a7e-3bee4f49a85a admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:21.347179 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-a0d54750-2e51-4d9b-af61-2396c5a14fe9 admin admin] The token cache is being invalidate because project bee867c79b4849958b17ca3ba6995d90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:25:21.350399 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-a0d54750-2e51-4d9b-af61-2396c5a14fe9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:21.350399 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-a0d54750-2e51-4d9b-af61-2396c5a14fe9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:21.355376 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 850/1680] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:25:20 2022] DELETE /identity/v3/projects/bee867c79b4849958b17ca3ba6995d90 => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:21.358278 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadbcf1-923d-4297-b52b-e354aa8479f2 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:21.359469 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadbcf1-923d-4297-b52b-e354aa8479f2 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:21.360154 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadbcf1-923d-4297-b52b-e354aa8479f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:21.405395 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-3cadbcf1-923d-4297-b52b-e354aa8479f2 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:21.407851 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-3cadbcf1-923d-4297-b52b-e354aa8479f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:21.482952 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35de9bb-a6b5-4390-8a7e-3bee4f49a85a admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:21.492294 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a2620-75b8-435c-b752-fd769d2650c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:21.493625 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a2620-75b8-435c-b752-fd769d2650c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:21.494319 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a2620-75b8-435c-b752-fd769d2650c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:21.494750 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a2620-75b8-435c-b752-fd769d2650c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:21.496177 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7a2620-75b8-435c-b752-fd769d2650c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:21.506411 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7a2620-75b8-435c-b752-fd769d2650c5 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:21.512457 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 851/1681] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:21.533780 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902824e3-472e-4d4a-96c3-8867dd0b89df None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:21.534121 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902824e3-472e-4d4a-96c3-8867dd0b89df None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:21.534734 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902824e3-472e-4d4a-96c3-8867dd0b89df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:21.541983 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:21.606409 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-902824e3-472e-4d4a-96c3-8867dd0b89df None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:21.610598 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-902824e3-472e-4d4a-96c3-8867dd0b89df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:21.627093 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 831/1682] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:25:21 2022] POST /identity/v3/users => generated 281 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:21.631668 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a2620-75b8-435c-b752-fd769d2650c5 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:21.633673 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-9e7a2620-75b8-435c-b752-fd769d2650c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:21.657590 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 852/1683] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:21.659455 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a2620-75b8-435c-b752-fd769d2650c5 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:21.662588 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902824e3-472e-4d4a-96c3-8867dd0b89df None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:21.676204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47907a32-3848-4bd5-98db-fc801a35ebb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:21.681091 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47907a32-3848-4bd5-98db-fc801a35ebb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:21.681091 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47907a32-3848-4bd5-98db-fc801a35ebb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:21.681091 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47907a32-3848-4bd5-98db-fc801a35ebb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:21.681091 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47907a32-3848-4bd5-98db-fc801a35ebb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:21.681372 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902824e3-472e-4d4a-96c3-8867dd0b89df None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:21.685719 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47907a32-3848-4bd5-98db-fc801a35ebb4 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:21.686928 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943c1ab9-622b-46d8-9823-0bb459aaac99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:21.687929 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c1ab9-622b-46d8-9823-0bb459aaac99 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:21.688279 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c1ab9-622b-46d8-9823-0bb459aaac99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:21.688665 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c1ab9-622b-46d8-9823-0bb459aaac99 admin admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:21.689596 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943c1ab9-622b-46d8-9823-0bb459aaac99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:25:21.690146 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943c1ab9-622b-46d8-9823-0bb459aaac99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:21.693082 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943c1ab9-622b-46d8-9823-0bb459aaac99 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:21.714822 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 832/1684] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:25:21.715709 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 853/1685] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:25:21.716989 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47907a32-3848-4bd5-98db-fc801a35ebb4 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:21.720193 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943c1ab9-622b-46d8-9823-0bb459aaac99 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:21.740165 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943c1ab9-622b-46d8-9823-0bb459aaac99 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:21.742923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47907a32-3848-4bd5-98db-fc801a35ebb4 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:21.779740 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d62f97-416a-48f4-a23a-dc9bda6715b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:21.780367 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d62f97-416a-48f4-a23a-dc9bda6715b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:21.780761 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d62f97-416a-48f4-a23a-dc9bda6715b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:21.781706 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d62f97-416a-48f4-a23a-dc9bda6715b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:21.782616 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d62f97-416a-48f4-a23a-dc9bda6715b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:21.784567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d62f97-416a-48f4-a23a-dc9bda6715b1 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:21.802358 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9432c5cd-9070-4f18-89fe-017c7232edd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:21.803038 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9432c5cd-9070-4f18-89fe-017c7232edd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:21.803323 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9432c5cd-9070-4f18-89fe-017c7232edd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:21.803596 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9432c5cd-9070-4f18-89fe-017c7232edd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:21.804280 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9432c5cd-9070-4f18-89fe-017c7232edd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:21.806306 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9432c5cd-9070-4f18-89fe-017c7232edd4 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:21.821420 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 833/1686] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:21.822713 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d62f97-416a-48f4-a23a-dc9bda6715b1 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:21.838421 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d62f97-416a-48f4-a23a-dc9bda6715b1 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:21.851584 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6546211-d7f1-4955-8a9e-6182516668c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:21.858288 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6546211-d7f1-4955-8a9e-6182516668c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:21.858700 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6546211-d7f1-4955-8a9e-6182516668c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:21.859097 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6546211-d7f1-4955-8a9e-6182516668c0 admin admin] PATH_INFO: `/v3/projects/a5015f41d4124d9998d47edb24f25660/users/0a25fd7b402e41068a7353de62e76d8c/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:21.899491 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6546211-d7f1-4955-8a9e-6182516668c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5015f41d4124d9998d47edb24f25660, user_id=0a25fd7b402e41068a7353de62e76d8c, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:21.902552 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6546211-d7f1-4955-8a9e-6182516668c0 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:21.943549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 854/1687] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:25:21 2022] POST /identity/v3/projects => generated 355 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:25:21.944968 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9432c5cd-9070-4f18-89fe-017c7232edd4 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:21.992633 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 834/1688] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:25:21 2022] PUT /identity/v3/projects/a5015f41d4124d9998d47edb24f25660/users/0a25fd7b402e41068a7353de62e76d8c/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:22.003221 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf50720-4c4d-41e9-9ad9-2e36b7f10dcf None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:22.003663 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf50720-4c4d-41e9-9ad9-2e36b7f10dcf None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:22.004036 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf50720-4c4d-41e9-9ad9-2e36b7f10dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:22.049099 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:22.083124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-3cf50720-4c4d-41e9-9ad9-2e36b7f10dcf None None] MFA Rules not processed for user `0a25fd7b402e41068a7353de62e76d8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:22.089215 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-3cf50720-4c4d-41e9-9ad9-2e36b7f10dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:22.162462 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9432c5cd-9070-4f18-89fe-017c7232edd4 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:22.223166 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 835/1689] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:22.224737 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf50720-4c4d-41e9-9ad9-2e36b7f10dcf None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:22.347845 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e624e5b-bee5-4fb0-a7ca-537b4068f46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:22.348528 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e624e5b-bee5-4fb0-a7ca-537b4068f46d admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:22.349263 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e624e5b-bee5-4fb0-a7ca-537b4068f46d admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:22.349646 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e624e5b-bee5-4fb0-a7ca-537b4068f46d admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:22.350225 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e624e5b-bee5-4fb0-a7ca-537b4068f46d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:22.352204 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e624e5b-bee5-4fb0-a7ca-537b4068f46d admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:22.423953 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf50720-4c4d-41e9-9ad9-2e36b7f10dcf None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:22.449707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae800be6-d440-4396-8d6e-3fcc46760bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:22.450592 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae800be6-d440-4396-8d6e-3fcc46760bb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:22.451017 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae800be6-d440-4396-8d6e-3fcc46760bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:22.451542 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae800be6-d440-4396-8d6e-3fcc46760bb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:22.458834 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae800be6-d440-4396-8d6e-3fcc46760bb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:22.462221 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae800be6-d440-4396-8d6e-3fcc46760bb6 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:22.489936 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 855/1690] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:25:21 2022] POST /identity/v3/users => generated 278 bytes in 543 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:22.489936 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e624e5b-bee5-4fb0-a7ca-537b4068f46d admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:22.493320 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-0e624e5b-bee5-4fb0-a7ca-537b4068f46d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:22.540312 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e624e5b-bee5-4fb0-a7ca-537b4068f46d admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:22.554416 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e28fc-9d22-4348-a18b-ab2d1cde723d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:22.557272 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e28fc-9d22-4348-a18b-ab2d1cde723d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:22.557567 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e28fc-9d22-4348-a18b-ab2d1cde723d admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:22.565267 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e28fc-9d22-4348-a18b-ab2d1cde723d admin admin] PATH_INFO: `/v3/projects/a41a81a283c744c2bd3276e1caa5d619` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:22.569806 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e28fc-9d22-4348-a18b-ab2d1cde723d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a41a81a283c744c2bd3276e1caa5d619)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:22.588715 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e28fc-9d22-4348-a18b-ab2d1cde723d admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:22.596819 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 836/1691] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:25:22 2022] POST /identity/v3/projects => generated 365 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:22.598589 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae800be6-d440-4396-8d6e-3fcc46760bb6 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:22.707393 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae800be6-d440-4396-8d6e-3fcc46760bb6 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:22.717122 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1de576-5276-47af-abd3-c5087c35bfd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:22.717775 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1de576-5276-47af-abd3-c5087c35bfd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:22.718082 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1de576-5276-47af-abd3-c5087c35bfd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:22.718363 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1de576-5276-47af-abd3-c5087c35bfd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:22.719368 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1de576-5276-47af-abd3-c5087c35bfd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:22.721088 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1de576-5276-47af-abd3-c5087c35bfd7 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:22.755676 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 837/1692] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:22.757111 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1de576-5276-47af-abd3-c5087c35bfd7 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:22.798186 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1de576-5276-47af-abd3-c5087c35bfd7 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:22.801248 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-e64e28fc-9d22-4348-a18b-ab2d1cde723d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a41a81a283c744c2bd3276e1caa5d619', 'request_id': 'req-e64e28fc-9d22-4348-a18b-ab2d1cde723d'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:22.822767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-e64e28fc-9d22-4348-a18b-ab2d1cde723d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a41a81a283c744c2bd3276e1caa5d619', 'request_id': 'req-e64e28fc-9d22-4348-a18b-ab2d1cde723d'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:22.850758 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-e64e28fc-9d22-4348-a18b-ab2d1cde723d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a41a81a283c744c2bd3276e1caa5d619', 'request_id': 'req-e64e28fc-9d22-4348-a18b-ab2d1cde723d'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:22.991938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-e64e28fc-9d22-4348-a18b-ab2d1cde723d admin admin] The token cache is being invalidate because project a41a81a283c744c2bd3276e1caa5d619 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:25:22.992191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-e64e28fc-9d22-4348-a18b-ab2d1cde723d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:22.993128 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-e64e28fc-9d22-4348-a18b-ab2d1cde723d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:22.994190 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea7718-9a50-490f-a157-9270119e19f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:22.995187 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea7718-9a50-490f-a157-9270119e19f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:22.995612 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea7718-9a50-490f-a157-9270119e19f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:22.995844 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 856/1693] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:25:22 2022] DELETE /identity/v3/projects/a41a81a283c744c2bd3276e1caa5d619 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:22.996180 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea7718-9a50-490f-a157-9270119e19f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:22.996759 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e28fc-9d22-4348-a18b-ab2d1cde723d admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:22.997623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ea7718-9a50-490f-a157-9270119e19f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:22.999775 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ea7718-9a50-490f-a157-9270119e19f4 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:23.005387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-e64e28fc-9d22-4348-a18b-ab2d1cde723d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:23.048871 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e28fc-9d22-4348-a18b-ab2d1cde723d admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:23.067795 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc732f-02f4-4354-968f-d628ec0cdba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:23.070716 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc732f-02f4-4354-968f-d628ec0cdba8 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:23.071024 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc732f-02f4-4354-968f-d628ec0cdba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:23.071302 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc732f-02f4-4354-968f-d628ec0cdba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:23.076657 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dc732f-02f4-4354-968f-d628ec0cdba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:23.079628 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dc732f-02f4-4354-968f-d628ec0cdba8 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:23.123114 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 857/1694] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:23.127517 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc732f-02f4-4354-968f-d628ec0cdba8 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:23.134187 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-c7dc732f-02f4-4354-968f-d628ec0cdba8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:23.166970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 838/1695] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:25:22 2022] POST /identity/v3/users => generated 289 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:23.168035 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea7718-9a50-490f-a157-9270119e19f4 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:23.186133 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc732f-02f4-4354-968f-d628ec0cdba8 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:23.186717 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea7718-9a50-490f-a157-9270119e19f4 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:23.196811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789415a6-7ccf-49d4-8ec9-107beed2fb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:23.197943 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789415a6-7ccf-49d4-8ec9-107beed2fb8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:23.198362 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789415a6-7ccf-49d4-8ec9-107beed2fb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:23.211524 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789415a6-7ccf-49d4-8ec9-107beed2fb8b admin admin] PATH_INFO: `/v3/projects/8ee5da8897ed438f9a0304baf9ace198/users/79a18e6676da4c6088ed9bd79843500a/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:23.217724 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e917a-c7ac-4df9-a30d-376a2539524b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:23.224779 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e917a-c7ac-4df9-a30d-376a2539524b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:23.225126 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e917a-c7ac-4df9-a30d-376a2539524b admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:23.225422 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e917a-c7ac-4df9-a30d-376a2539524b admin admin] PATH_INFO: `/v3/users/6da90312772a4cd6976146c834b993fc` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:23.230441 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:23.232015 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e917a-c7ac-4df9-a30d-376a2539524b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6da90312772a4cd6976146c834b993fc)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:23.237344 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e917a-c7ac-4df9-a30d-376a2539524b admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:23.258667 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789415a6-7ccf-49d4-8ec9-107beed2fb8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ee5da8897ed438f9a0304baf9ace198, user_id=79a18e6676da4c6088ed9bd79843500a, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:23.262203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789415a6-7ccf-49d4-8ec9-107beed2fb8b admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:23.408252 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 839/1696] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:25:23 2022] PUT /identity/v3/projects/8ee5da8897ed438f9a0304baf9ace198/users/79a18e6676da4c6088ed9bd79843500a/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:23.409182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789415a6-7ccf-49d4-8ec9-107beed2fb8b admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:23.423795 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-789415a6-7ccf-49d4-8ec9-107beed2fb8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:23.518858 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-b48e917a-c7ac-4df9-a30d-376a2539524b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6da90312772a4cd6976146c834b993fc'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:23.541058 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-b48e917a-c7ac-4df9-a30d-376a2539524b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6da90312772a4cd6976146c834b993fc'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:23.542724 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-b48e917a-c7ac-4df9-a30d-376a2539524b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6da90312772a4cd6976146c834b993fc'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:23.544853 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-b48e917a-c7ac-4df9-a30d-376a2539524b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6da90312772a4cd6976146c834b993fc'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:23.582832 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-b48e917a-c7ac-4df9-a30d-376a2539524b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6da90312772a4cd6976146c834b993fc'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:23.593323 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789415a6-7ccf-49d4-8ec9-107beed2fb8b admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:23.603537 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb0ded9-837b-44e8-a5d2-6cbeacf5376a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:23.604208 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0ded9-837b-44e8-a5d2-6cbeacf5376a admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:23.605308 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0ded9-837b-44e8-a5d2-6cbeacf5376a admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:23.605601 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0ded9-837b-44e8-a5d2-6cbeacf5376a admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:23.606474 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb0ded9-837b-44e8-a5d2-6cbeacf5376a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:23.610292 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb0ded9-837b-44e8-a5d2-6cbeacf5376a admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:23.622170 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 840/1697] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:23.623676 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2d7af-3e21-43e6-a5e0-b5cd690ab117 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:23.624140 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2d7af-3e21-43e6-a5e0-b5cd690ab117 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:23.624674 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2d7af-3e21-43e6-a5e0-b5cd690ab117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:23.669597 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-0cf2d7af-3e21-43e6-a5e0-b5cd690ab117 None None] MFA Rules not processed for user `3bbe964ead724a4da59620786b5706d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:23.671486 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-0cf2d7af-3e21-43e6-a5e0-b5cd690ab117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:23.737862 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 858/1698] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:25:23 2022] DELETE /identity/v3/users/6da90312772a4cd6976146c834b993fc => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:23.742806 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b6be4-1573-46b2-8508-3ca7af690b59 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:23.743739 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b6be4-1573-46b2-8508-3ca7af690b59 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:23.744216 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b6be4-1573-46b2-8508-3ca7af690b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:23.840583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-4f5b6be4-1573-46b2-8508-3ca7af690b59 None None] MFA Rules not processed for user `79a18e6676da4c6088ed9bd79843500a`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:23.842504 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-4f5b6be4-1573-46b2-8508-3ca7af690b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:23.861645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 841/1699] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:23.862848 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf2d7af-3e21-43e6-a5e0-b5cd690ab117 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:23.887147 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-0cf2d7af-3e21-43e6-a5e0-b5cd690ab117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:23.895076 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:23.931468 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf2d7af-3e21-43e6-a5e0-b5cd690ab117 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:23.995596 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 859/1700] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:23 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:23.996716 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f0950-f0c1-4277-993f-7cd0009d2240 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:23.999595 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f0950-f0c1-4277-993f-7cd0009d2240 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:23.999986 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f0950-f0c1-4277-993f-7cd0009d2240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:24.068689 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-980f0950-f0c1-4277-993f-7cd0009d2240 None None] MFA Rules not processed for user `3bbe964ead724a4da59620786b5706d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:24.072671 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-980f0950-f0c1-4277-993f-7cd0009d2240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:24.110663 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f07630-1f16-4b15-9695-c3b1f5a8b4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:24.115607 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f07630-1f16-4b15-9695-c3b1f5a8b4b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:24.116450 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f07630-1f16-4b15-9695-c3b1f5a8b4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:24.118457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f07630-1f16-4b15-9695-c3b1f5a8b4b8 admin admin] PATH_INFO: `/v3/projects/86b92b44328c4c41aae7fb4728305ad8/users/3b14c825c9d84a7993e6ba2a6f1c0bd2/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:24.172897 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f07630-1f16-4b15-9695-c3b1f5a8b4b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86b92b44328c4c41aae7fb4728305ad8, user_id=3b14c825c9d84a7993e6ba2a6f1c0bd2, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:24.177412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f07630-1f16-4b15-9695-c3b1f5a8b4b8 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:24.265065 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 860/1701] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:24.301344 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f0950-f0c1-4277-993f-7cd0009d2240 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:24.316584 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 842/1702] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:25:23 2022] PUT /identity/v3/projects/86b92b44328c4c41aae7fb4728305ad8/users/3b14c825c9d84a7993e6ba2a6f1c0bd2/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:24.319834 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f07630-1f16-4b15-9695-c3b1f5a8b4b8 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:24.320182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-71f07630-1f16-4b15-9695-c3b1f5a8b4b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:24.346497 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-980f0950-f0c1-4277-993f-7cd0009d2240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:24.444254 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f0950-f0c1-4277-993f-7cd0009d2240 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:24.653879 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022be9b6-754a-47e2-8f7f-b3dafe14e626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:24.655250 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022be9b6-754a-47e2-8f7f-b3dafe14e626 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:24.655684 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022be9b6-754a-47e2-8f7f-b3dafe14e626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:24.656094 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022be9b6-754a-47e2-8f7f-b3dafe14e626 admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:24.657792 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022be9b6-754a-47e2-8f7f-b3dafe14e626 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:24.663012 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022be9b6-754a-47e2-8f7f-b3dafe14e626 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:24.669584 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f07630-1f16-4b15-9695-c3b1f5a8b4b8 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:24.687890 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38937d2-e92d-4e6c-ac95-8d8bd1d94c8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:24.689023 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38937d2-e92d-4e6c-ac95-8d8bd1d94c8f service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:24.689453 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38937d2-e92d-4e6c-ac95-8d8bd1d94c8f service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:24.689893 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38937d2-e92d-4e6c-ac95-8d8bd1d94c8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:24.775773 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 861/1703] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:25:24 2022] POST /identity/v3/projects => generated 355 bytes in 510 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:24.779880 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022be9b6-754a-47e2-8f7f-b3dafe14e626 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:24.844754 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38937d2-e92d-4e6c-ac95-8d8bd1d94c8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:24.847608 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38937d2-e92d-4e6c-ac95-8d8bd1d94c8f service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:24.906136 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 843/1704] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:24.912054 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38937d2-e92d-4e6c-ac95-8d8bd1d94c8f service nova] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:24.924360 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d38937d2-e92d-4e6c-ac95-8d8bd1d94c8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:24.930856 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022be9b6-754a-47e2-8f7f-b3dafe14e626 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:24.949109 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42023525-314a-4009-8fd1-f8f7b83d8808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:24.949762 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42023525-314a-4009-8fd1-f8f7b83d8808 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:24.950048 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42023525-314a-4009-8fd1-f8f7b83d8808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:24.950456 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42023525-314a-4009-8fd1-f8f7b83d8808 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:24.951185 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42023525-314a-4009-8fd1-f8f7b83d8808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:24.952843 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42023525-314a-4009-8fd1-f8f7b83d8808 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:24.970473 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38937d2-e92d-4e6c-ac95-8d8bd1d94c8f service nova] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:24.982587 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d098252-7edf-4c5f-a349-486a35fbd795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:24.986916 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 862/1705] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:24.988153 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42023525-314a-4009-8fd1-f8f7b83d8808 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:25.007539 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d098252-7edf-4c5f-a349-486a35fbd795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:25.008043 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d098252-7edf-4c5f-a349-486a35fbd795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:25.008574 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d098252-7edf-4c5f-a349-486a35fbd795 admin admin] PATH_INFO: `/v3/projects/84346e3c882347dfb36ba372631d5e35` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:25.010465 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d098252-7edf-4c5f-a349-486a35fbd795 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84346e3c882347dfb36ba372631d5e35)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:25.012943 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d098252-7edf-4c5f-a349-486a35fbd795 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:25.013254 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42023525-314a-4009-8fd1-f8f7b83d8808 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:25.026981 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec10ca-352b-425c-ba6f-5d993e185252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:25.035265 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec10ca-352b-425c-ba6f-5d993e185252 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:25.035265 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec10ca-352b-425c-ba6f-5d993e185252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:25.035265 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec10ca-352b-425c-ba6f-5d993e185252 admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:25.035265 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aec10ca-352b-425c-ba6f-5d993e185252 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:25.035265 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aec10ca-352b-425c-ba6f-5d993e185252 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:25.187307 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 863/1706] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:25:24 2022] POST /identity/v3/users => generated 284 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:25.188456 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec10ca-352b-425c-ba6f-5d993e185252 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:25.217183 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec10ca-352b-425c-ba6f-5d993e185252 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:25.264403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7d098252-7edf-4c5f-a349-486a35fbd795 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84346e3c882347dfb36ba372631d5e35', 'request_id': 'req-7d098252-7edf-4c5f-a349-486a35fbd795'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:25.283316 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7d098252-7edf-4c5f-a349-486a35fbd795 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84346e3c882347dfb36ba372631d5e35', 'request_id': 'req-7d098252-7edf-4c5f-a349-486a35fbd795'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:25.345720 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:25.381398 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19686210-462f-40a6-a23f-91bd0eef6f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:25.382112 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19686210-462f-40a6-a23f-91bd0eef6f45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:25.382411 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19686210-462f-40a6-a23f-91bd0eef6f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:25.383313 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19686210-462f-40a6-a23f-91bd0eef6f45 admin admin] PATH_INFO: `/v3/projects/86b92b44328c4c41aae7fb4728305ad8/users/3b14c825c9d84a7993e6ba2a6f1c0bd2/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:25.391742 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19686210-462f-40a6-a23f-91bd0eef6f45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86b92b44328c4c41aae7fb4728305ad8, user_id=3b14c825c9d84a7993e6ba2a6f1c0bd2, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:25.399228 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19686210-462f-40a6-a23f-91bd0eef6f45 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:25.402834 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7d098252-7edf-4c5f-a349-486a35fbd795 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84346e3c882347dfb36ba372631d5e35', 'request_id': 'req-7d098252-7edf-4c5f-a349-486a35fbd795'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:25.432496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7d098252-7edf-4c5f-a349-486a35fbd795 admin admin] The token cache is being invalidate because project 84346e3c882347dfb36ba372631d5e35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:25:25.432780 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7d098252-7edf-4c5f-a349-486a35fbd795 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:25.434935 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7d098252-7edf-4c5f-a349-486a35fbd795 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:25.438039 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 864/1707] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:25:25 2022] PUT /identity/v3/projects/86b92b44328c4c41aae7fb4728305ad8/users/3b14c825c9d84a7993e6ba2a6f1c0bd2/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:25.438941 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19686210-462f-40a6-a23f-91bd0eef6f45 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:25.439747 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 844/1708] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:25:24 2022] DELETE /identity/v3/projects/84346e3c882347dfb36ba372631d5e35 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:25.449196 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa25beca-b91c-4ee0-94cb-93177aae2309 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:25.451566 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa25beca-b91c-4ee0-94cb-93177aae2309 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:25.451855 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa25beca-b91c-4ee0-94cb-93177aae2309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:25.452751 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-19686210-462f-40a6-a23f-91bd0eef6f45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:25.567017 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19686210-462f-40a6-a23f-91bd0eef6f45 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:25.578687 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e08b5-2891-4eed-9ca0-b8033b5f592c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:25.579556 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e08b5-2891-4eed-9ca0-b8033b5f592c admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:25.580806 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e08b5-2891-4eed-9ca0-b8033b5f592c admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:25.581276 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e08b5-2891-4eed-9ca0-b8033b5f592c admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:25.582149 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087e08b5-2891-4eed-9ca0-b8033b5f592c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:25.589731 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087e08b5-2891-4eed-9ca0-b8033b5f592c admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:25.594798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:25.611422 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-aa25beca-b91c-4ee0-94cb-93177aae2309 None None] MFA Rules not processed for user `99dc94fff06f42c39f3750484f6d8e9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:25.613888 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 865/1709] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:25.617743 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-aa25beca-b91c-4ee0-94cb-93177aae2309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:25.618406 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26bbb9-4041-41aa-9399-9ada5c75a567 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:25.618737 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26bbb9-4041-41aa-9399-9ada5c75a567 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:25.622563 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26bbb9-4041-41aa-9399-9ada5c75a567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:25.678297 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-1f26bbb9-4041-41aa-9399-9ada5c75a567 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:25.679613 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-1f26bbb9-4041-41aa-9399-9ada5c75a567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:25.701377 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 866/1710] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:25.703640 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0c214-a826-43cf-aa4c-8298cbc36908 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:25.705700 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0c214-a826-43cf-aa4c-8298cbc36908 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:25.706115 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0c214-a826-43cf-aa4c-8298cbc36908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:25.785122 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 845/1711] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:25.785122 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa25beca-b91c-4ee0-94cb-93177aae2309 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:25.791382 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-25a0c214-a826-43cf-aa4c-8298cbc36908 None None] MFA Rules not processed for user `3b14c825c9d84a7993e6ba2a6f1c0bd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:25.792993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-25a0c214-a826-43cf-aa4c-8298cbc36908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:25.815583 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa25beca-b91c-4ee0-94cb-93177aae2309 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:25.853181 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd22eee-3628-422d-a2ac-fe5f6ec79916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:25.855848 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd22eee-3628-422d-a2ac-fe5f6ec79916 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:25.856138 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd22eee-3628-422d-a2ac-fe5f6ec79916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:25.856484 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd22eee-3628-422d-a2ac-fe5f6ec79916 admin admin] PATH_INFO: `/v3/projects/f8c178e5cd7c4a9293f9689b8a7c1161/users/17e59984bc7145cebe0c0ab15f68549a/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:25.910419 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 867/1712] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:25 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:25.911196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e49b4-c7bb-47ea-895d-681435b1d441 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:25.912741 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e49b4-c7bb-47ea-895d-681435b1d441 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:25.913551 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e49b4-c7bb-47ea-895d-681435b1d441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:25.928610 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd22eee-3628-422d-a2ac-fe5f6ec79916 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8c178e5cd7c4a9293f9689b8a7c1161, user_id=17e59984bc7145cebe0c0ab15f68549a, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:25.936706 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd22eee-3628-422d-a2ac-fe5f6ec79916 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:25.978232 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-8f4e49b4-c7bb-47ea-895d-681435b1d441 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:25.980330 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-8f4e49b4-c7bb-47ea-895d-681435b1d441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:25.995306 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 868/1713] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:25.996340 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e49b4-c7bb-47ea-895d-681435b1d441 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:26.003167 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-8f4e49b4-c7bb-47ea-895d-681435b1d441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:26.031972 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 846/1714] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:25:25 2022] PUT /identity/v3/projects/f8c178e5cd7c4a9293f9689b8a7c1161/users/17e59984bc7145cebe0c0ab15f68549a/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:26.037588 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd22eee-3628-422d-a2ac-fe5f6ec79916 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:26.040554 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ccd22eee-3628-422d-a2ac-fe5f6ec79916 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:26.131613 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e49b4-c7bb-47ea-895d-681435b1d441 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:26.251764 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd22eee-3628-422d-a2ac-fe5f6ec79916 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:26.271124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd4078-b0be-44b8-b0d4-4942d9d8a400 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:26.271420 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd4078-b0be-44b8-b0d4-4942d9d8a400 service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:26.271694 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd4078-b0be-44b8-b0d4-4942d9d8a400 service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:26.271970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd4078-b0be-44b8-b0d4-4942d9d8a400 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:26.278716 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8646f91c-a6b2-4e43-894e-810b93388aa4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:26.279367 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8646f91c-a6b2-4e43-894e-810b93388aa4 service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:26.280008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8646f91c-a6b2-4e43-894e-810b93388aa4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:26.280008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8646f91c-a6b2-4e43-894e-810b93388aa4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:26.372829 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:26.386338 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd4078-b0be-44b8-b0d4-4942d9d8a400 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:26.390093 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd4078-b0be-44b8-b0d4-4942d9d8a400 service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:26.415042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 847/1715] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:26.415956 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd4078-b0be-44b8-b0d4-4942d9d8a400 service nova] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:26.425670 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8646f91c-a6b2-4e43-894e-810b93388aa4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:26.428089 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8646f91c-a6b2-4e43-894e-810b93388aa4 service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:26.455404 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 869/1716] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:25 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:26.456192 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8646f91c-a6b2-4e43-894e-810b93388aa4 service cinder] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:26.581236 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd4078-b0be-44b8-b0d4-4942d9d8a400 service nova] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:26.595471 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd498ae-2863-40ad-978b-957825d06865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:26.596534 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd498ae-2863-40ad-978b-957825d06865 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:26.607464 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd498ae-2863-40ad-978b-957825d06865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:26.607464 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd498ae-2863-40ad-978b-957825d06865 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:26.607464 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd498ae-2863-40ad-978b-957825d06865 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:25:26.607464 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd498ae-2863-40ad-978b-957825d06865 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:26.607464 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd498ae-2863-40ad-978b-957825d06865 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:26.626685 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 848/1717] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:25:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:25:26.632586 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4470bc-60cf-4b85-9c6f-b823dc73fa79 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:26.636333 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4470bc-60cf-4b85-9c6f-b823dc73fa79 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:26.636333 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4470bc-60cf-4b85-9c6f-b823dc73fa79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:26.646064 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8646f91c-a6b2-4e43-894e-810b93388aa4 service cinder] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:26.658241 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c119c-88fe-45c5-8a94-a6ff168d020b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:26.659513 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c119c-88fe-45c5-8a94-a6ff168d020b admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:26.659999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c119c-88fe-45c5-8a94-a6ff168d020b admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:26.660528 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c119c-88fe-45c5-8a94-a6ff168d020b admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:26.662102 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91c119c-88fe-45c5-8a94-a6ff168d020b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:26.664309 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91c119c-88fe-45c5-8a94-a6ff168d020b admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:26.680131 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 870/1718] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:26.687929 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c119c-88fe-45c5-8a94-a6ff168d020b admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:26.696274 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c119c-88fe-45c5-8a94-a6ff168d020b admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:26.701759 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c69a76-65ea-4d78-ac7e-f435284e4d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:26.702753 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c69a76-65ea-4d78-ac7e-f435284e4d5e admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:26.703378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c69a76-65ea-4d78-ac7e-f435284e4d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:26.703909 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c69a76-65ea-4d78-ac7e-f435284e4d5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:26.704824 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c69a76-65ea-4d78-ac7e-f435284e4d5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:26.707466 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c69a76-65ea-4d78-ac7e-f435284e4d5e admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:26.772983 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-6f4470bc-60cf-4b85-9c6f-b823dc73fa79 None None] MFA Rules not processed for user `17e3b509a2de4159bf234594573f5e18`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:26.774403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-6f4470bc-60cf-4b85-9c6f-b823dc73fa79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:26.924980 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 871/1719] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:25:26 2022] POST /identity/v3/projects => generated 385 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:26.924980 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c69a76-65ea-4d78-ac7e-f435284e4d5e admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:26.954135 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 849/1720] 10.222.0.26 () {58 vars in 1009 bytes} [Fri Aug 5 14:25:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:26.958938 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401e0d2-29d8-4014-b0ec-98e80c422837 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:26.960289 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401e0d2-29d8-4014-b0ec-98e80c422837 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:26.960701 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401e0d2-29d8-4014-b0ec-98e80c422837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:27.038102 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-f401e0d2-29d8-4014-b0ec-98e80c422837 None None] MFA Rules not processed for user `084b18670c2a4c1396d5912d130b2bac`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:27.039429 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f401e0d2-29d8-4014-b0ec-98e80c422837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:27.066004 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:27.204218 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c69a76-65ea-4d78-ac7e-f435284e4d5e admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:27.218777 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c292c652-6006-40b2-9390-e799bd81d7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:27.221165 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292c652-6006-40b2-9390-e799bd81d7bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:27.221462 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292c652-6006-40b2-9390-e799bd81d7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:27.221750 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292c652-6006-40b2-9390-e799bd81d7bd admin admin] PATH_INFO: `/v3/projects/f8c178e5cd7c4a9293f9689b8a7c1161/users/17e59984bc7145cebe0c0ab15f68549a/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:27.226719 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c292c652-6006-40b2-9390-e799bd81d7bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8c178e5cd7c4a9293f9689b8a7c1161, user_id=17e59984bc7145cebe0c0ab15f68549a, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:27.232490 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 850/1721] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:26 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:27.233206 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f401e0d2-29d8-4014-b0ec-98e80c422837 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:27.243742 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c292c652-6006-40b2-9390-e799bd81d7bd admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:27.273438 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 872/1722] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:25:26 2022] PUT /identity/v3/projects/f8c178e5cd7c4a9293f9689b8a7c1161/users/17e59984bc7145cebe0c0ab15f68549a/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:27.274994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c292c652-6006-40b2-9390-e799bd81d7bd admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:27.371161 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f401e0d2-29d8-4014-b0ec-98e80c422837 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:27.395064 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:27.476665 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c292c652-6006-40b2-9390-e799bd81d7bd admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:27.482090 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9108f89b-1cfe-45f5-9fdf-2451d650bfb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:27.482724 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108f89b-1cfe-45f5-9fdf-2451d650bfb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:27.483115 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108f89b-1cfe-45f5-9fdf-2451d650bfb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:27.483397 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108f89b-1cfe-45f5-9fdf-2451d650bfb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:27.484092 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9108f89b-1cfe-45f5-9fdf-2451d650bfb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:27.486616 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9108f89b-1cfe-45f5-9fdf-2451d650bfb3 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:27.557873 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a9d2cd-3d19-4a95-bb07-91a47b1dcda3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:27.558742 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a9d2cd-3d19-4a95-bb07-91a47b1dcda3 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:27.559190 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a9d2cd-3d19-4a95-bb07-91a47b1dcda3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:27.559641 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a9d2cd-3d19-4a95-bb07-91a47b1dcda3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:27.573095 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-b2a9d2cd-3d19-4a95-bb07-91a47b1dcda3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:27.617758 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 873/1723] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:25:27 2022] POST /identity/v3/users => generated 293 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:27.619810 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd856a-76ff-4d2d-bce4-455bca6d9639 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:27.621310 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd856a-76ff-4d2d-bce4-455bca6d9639 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:27.621887 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd856a-76ff-4d2d-bce4-455bca6d9639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:27.681658 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-bddd856a-76ff-4d2d-bce4-455bca6d9639 None None] MFA Rules not processed for user `17e59984bc7145cebe0c0ab15f68549a`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:27.684506 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-bddd856a-76ff-4d2d-bce4-455bca6d9639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:27.723013 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a9d2cd-3d19-4a95-bb07-91a47b1dcda3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:27.725116 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a9d2cd-3d19-4a95-bb07-91a47b1dcda3 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:27.758724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 851/1724] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:27.759876 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a9d2cd-3d19-4a95-bb07-91a47b1dcda3 service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:27.843818 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 874/1725] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:27 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:27.844805 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd856a-76ff-4d2d-bce4-455bca6d9639 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:27.848926 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-bddd856a-76ff-4d2d-bce4-455bca6d9639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:27.880007 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd856a-76ff-4d2d-bce4-455bca6d9639 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:27.891186 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf6936-e0eb-48f1-a345-ee728468f8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:27.891859 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf6936-e0eb-48f1-a345-ee728468f8f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:27.892142 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf6936-e0eb-48f1-a345-ee728468f8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:27.892454 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf6936-e0eb-48f1-a345-ee728468f8f1 admin admin] PATH_INFO: `/v3/users/b5c7a069800d476d9df8f945120ad7ea` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:27.895312 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabf6936-e0eb-48f1-a345-ee728468f8f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5c7a069800d476d9df8f945120ad7ea)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:27.897573 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabf6936-e0eb-48f1-a345-ee728468f8f1 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:28.001309 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a9d2cd-3d19-4a95-bb07-91a47b1dcda3 service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:28.010526 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517835d-34a3-43d8-8fa0-77c3c5aa31f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:28.011173 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517835d-34a3-43d8-8fa0-77c3c5aa31f8 service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:28.011450 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517835d-34a3-43d8-8fa0-77c3c5aa31f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:28.011721 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517835d-34a3-43d8-8fa0-77c3c5aa31f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:28.162743 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-eabf6936-e0eb-48f1-a345-ee728468f8f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5c7a069800d476d9df8f945120ad7ea'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:28.162850 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8517835d-34a3-43d8-8fa0-77c3c5aa31f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:28.170423 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8517835d-34a3-43d8-8fa0-77c3c5aa31f8 service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:28.177931 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-eabf6936-e0eb-48f1-a345-ee728468f8f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5c7a069800d476d9df8f945120ad7ea'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:28.180860 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-eabf6936-e0eb-48f1-a345-ee728468f8f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5c7a069800d476d9df8f945120ad7ea'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:28.181517 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-eabf6936-e0eb-48f1-a345-ee728468f8f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5c7a069800d476d9df8f945120ad7ea'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:28.185435 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 852/1726] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:27 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:28.186552 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517835d-34a3-43d8-8fa0-77c3c5aa31f8 service nova] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:28.188573 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8517835d-34a3-43d8-8fa0-77c3c5aa31f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:28.215164 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-eabf6936-e0eb-48f1-a345-ee728468f8f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5c7a069800d476d9df8f945120ad7ea'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:28.319203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517835d-34a3-43d8-8fa0-77c3c5aa31f8 service nova] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:28.324576 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a53761a-bc6d-44f4-9da6-31b7294d6ba3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dddd7eea67f34f0cbb5f6a1cd85f1fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:28.325226 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a53761a-bc6d-44f4-9da6-31b7294d6ba3 service swift] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:28.325504 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a53761a-bc6d-44f4-9da6-31b7294d6ba3 service swift] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:28.326687 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a53761a-bc6d-44f4-9da6-31b7294d6ba3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:28.328010 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5a53761a-bc6d-44f4-9da6-31b7294d6ba3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:28.336600 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 875/1727] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:25:27 2022] DELETE /identity/v3/users/b5c7a069800d476d9df8f945120ad7ea => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:28.337965 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf6936-e0eb-48f1-a345-ee728468f8f1 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:28.339452 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-eabf6936-e0eb-48f1-a345-ee728468f8f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:28.399755 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:28.411963 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a53761a-bc6d-44f4-9da6-31b7294d6ba3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:28.414380 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a53761a-bc6d-44f4-9da6-31b7294d6ba3 service swift] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:28.425499 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 853/1728] 10.222.0.26 () {60 vars in 1384 bytes} [Fri Aug 5 14:25:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 14:25:28.425499 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a53761a-bc6d-44f4-9da6-31b7294d6ba3 service swift] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:28.440888 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5a53761a-bc6d-44f4-9da6-31b7294d6ba3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:28.517340 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a53761a-bc6d-44f4-9da6-31b7294d6ba3 service swift] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:28.636140 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:28.656769 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf6936-e0eb-48f1-a345-ee728468f8f1 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:28.682780 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf2be0-7412-4f2e-9a40-18e2e18f4af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:28.684153 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf2be0-7412-4f2e-9a40-18e2e18f4af6 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:28.698950 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf2be0-7412-4f2e-9a40-18e2e18f4af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:28.698950 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf2be0-7412-4f2e-9a40-18e2e18f4af6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:28.698950 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf2be0-7412-4f2e-9a40-18e2e18f4af6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:28.699927 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf2be0-7412-4f2e-9a40-18e2e18f4af6 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:28.740021 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 876/1729] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:28.748052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca310e-cecf-422e-8061-9bbe7f429f97 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:28.748364 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca310e-cecf-422e-8061-9bbe7f429f97 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:28.750802 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca310e-cecf-422e-8061-9bbe7f429f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:28.757202 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42730dcc-08bb-4591-9429-7e44ffbf9d14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:28.760661 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42730dcc-08bb-4591-9429-7e44ffbf9d14 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:28.760964 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42730dcc-08bb-4591-9429-7e44ffbf9d14 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:28.761246 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42730dcc-08bb-4591-9429-7e44ffbf9d14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:28.772239 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-42730dcc-08bb-4591-9429-7e44ffbf9d14 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:28.857070 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-1eca310e-cecf-422e-8061-9bbe7f429f97 None None] MFA Rules not processed for user `695e37560f8f495e987d1bd17acc2680`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:28.862325 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-1eca310e-cecf-422e-8061-9bbe7f429f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:28.964082 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42730dcc-08bb-4591-9429-7e44ffbf9d14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:28.966181 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42730dcc-08bb-4591-9429-7e44ffbf9d14 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:29.010572 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 854/1730] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:28 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:29.028167 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec87fb0-f9ad-4d1f-b300-3201563b9bd8 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:29.028505 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec87fb0-f9ad-4d1f-b300-3201563b9bd8 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:29.028779 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec87fb0-f9ad-4d1f-b300-3201563b9bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:29.081677 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 877/1731] 10.222.0.13 () {58 vars in 1015 bytes} [Fri Aug 5 14:25:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:29.082973 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eca310e-cecf-422e-8061-9bbe7f429f97 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:29.120321 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-3ec87fb0-f9ad-4d1f-b300-3201563b9bd8 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:29.121250 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eca310e-cecf-422e-8061-9bbe7f429f97 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:29.125765 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-3ec87fb0-f9ad-4d1f-b300-3201563b9bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:29.141174 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce3ab26-4055-4319-ae12-eef4d5c8b454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:29.143321 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3ab26-4055-4319-ae12-eef4d5c8b454 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:29.145444 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3ab26-4055-4319-ae12-eef4d5c8b454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:29.156145 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3ab26-4055-4319-ae12-eef4d5c8b454 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:29.189876 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce3ab26-4055-4319-ae12-eef4d5c8b454 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:29.189876 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce3ab26-4055-4319-ae12-eef4d5c8b454 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:29.190651 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 855/1732] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:29.195519 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec87fb0-f9ad-4d1f-b300-3201563b9bd8 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:29.209298 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 878/1733] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:29 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:29.209298 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce3ab26-4055-4319-ae12-eef4d5c8b454 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:29.210111 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-3ec87fb0-f9ad-4d1f-b300-3201563b9bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:29.213371 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce3ab26-4055-4319-ae12-eef4d5c8b454 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:29.219082 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bec0f5f-76c7-4c98-a3aa-d4c6eace0841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:29.219833 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec0f5f-76c7-4c98-a3aa-d4c6eace0841 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:29.220117 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec0f5f-76c7-4c98-a3aa-d4c6eace0841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:29.220390 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec0f5f-76c7-4c98-a3aa-d4c6eace0841 admin admin] PATH_INFO: `/v3/projects/c1996206308c4dc68cee8ef3ed4a5dfb/users/2d3c197b8f734f549589839633dc2359/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:29.270096 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec87fb0-f9ad-4d1f-b300-3201563b9bd8 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:29.286470 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9133c31-5dac-4555-a182-77c57a0d6df2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:29.287316 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9133c31-5dac-4555-a182-77c57a0d6df2 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:29.289404 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9133c31-5dac-4555-a182-77c57a0d6df2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:29.290889 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9133c31-5dac-4555-a182-77c57a0d6df2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:29.299339 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bec0f5f-76c7-4c98-a3aa-d4c6eace0841 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1996206308c4dc68cee8ef3ed4a5dfb, user_id=2d3c197b8f734f549589839633dc2359, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:29.308715 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bec0f5f-76c7-4c98-a3aa-d4c6eace0841 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:29.313678 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9133c31-5dac-4555-a182-77c57a0d6df2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:29.315145 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9133c31-5dac-4555-a182-77c57a0d6df2 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:29.363147 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 856/1734] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:29.370497 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9133c31-5dac-4555-a182-77c57a0d6df2 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:29.376924 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c9133c31-5dac-4555-a182-77c57a0d6df2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:29.423797 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9133c31-5dac-4555-a182-77c57a0d6df2 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:29.438016 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 879/1735] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:25:29 2022] PUT /identity/v3/projects/c1996206308c4dc68cee8ef3ed4a5dfb/users/2d3c197b8f734f549589839633dc2359/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:29.438016 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bec0f5f-76c7-4c98-a3aa-d4c6eace0841 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:29.457498 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:29.540173 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a89ce9-1212-4ffe-8e37-9cd1a3ecb76c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:29.541183 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a89ce9-1212-4ffe-8e37-9cd1a3ecb76c service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:29.541902 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a89ce9-1212-4ffe-8e37-9cd1a3ecb76c service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:29.542401 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a89ce9-1212-4ffe-8e37-9cd1a3ecb76c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:29.700299 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bec0f5f-76c7-4c98-a3aa-d4c6eace0841 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:29.721171 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8b83c-d6a9-4427-a66a-80ef78f9dd44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:29.721942 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8b83c-d6a9-4427-a66a-80ef78f9dd44 service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:29.722178 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8b83c-d6a9-4427-a66a-80ef78f9dd44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:29.722462 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8b83c-d6a9-4427-a66a-80ef78f9dd44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:29.731339 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-d5f8b83c-d6a9-4427-a66a-80ef78f9dd44 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:29.757313 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a89ce9-1212-4ffe-8e37-9cd1a3ecb76c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:29.758799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a89ce9-1212-4ffe-8e37-9cd1a3ecb76c service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:29.804358 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 857/1736] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:29.805687 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a086075-e842-4eb1-a70d-e073cd9ed9bc None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:29.815687 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a086075-e842-4eb1-a70d-e073cd9ed9bc None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:29.816201 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a086075-e842-4eb1-a70d-e073cd9ed9bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:29.920924 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-6a086075-e842-4eb1-a70d-e073cd9ed9bc None None] MFA Rules not processed for user `2d3c197b8f734f549589839633dc2359`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:29.924053 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-6a086075-e842-4eb1-a70d-e073cd9ed9bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:30.016636 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f8b83c-d6a9-4427-a66a-80ef78f9dd44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:30.019524 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f8b83c-d6a9-4427-a66a-80ef78f9dd44 service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:30.073768 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 880/1737] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:29 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 644 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:30.075536 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 858/1738] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:30.084767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8b83c-d6a9-4427-a66a-80ef78f9dd44 service cinder] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:30.097842 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8b83c-d6a9-4427-a66a-80ef78f9dd44 service cinder] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:30.105367 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ae81b-a6d5-4b40-a22f-449cceb77025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:30.111859 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ae81b-a6d5-4b40-a22f-449cceb77025 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:30.113208 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ae81b-a6d5-4b40-a22f-449cceb77025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:30.113541 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ae81b-a6d5-4b40-a22f-449cceb77025 admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:30.115383 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26ae81b-a6d5-4b40-a22f-449cceb77025 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:30.119861 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26ae81b-a6d5-4b40-a22f-449cceb77025 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:30.191963 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:30.530433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 881/1739] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:25:30 2022] POST /identity/v3/projects => generated 387 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:30.534682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a086075-e842-4eb1-a70d-e073cd9ed9bc None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:30.547368 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a086075-e842-4eb1-a70d-e073cd9ed9bc None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:30.744372 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a10b60-96ef-4d73-9d3d-e092b85f4e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:30.746007 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a10b60-96ef-4d73-9d3d-e092b85f4e82 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:30.746805 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a10b60-96ef-4d73-9d3d-e092b85f4e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:30.747200 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a10b60-96ef-4d73-9d3d-e092b85f4e82 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:30.749960 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a10b60-96ef-4d73-9d3d-e092b85f4e82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:30.755235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a10b60-96ef-4d73-9d3d-e092b85f4e82 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:30.847960 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:30.977695 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 859/1740] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:25:30 2022] POST /identity/v3/users => generated 300 bytes in 441 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:30.978932 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ae81b-a6d5-4b40-a22f-449cceb77025 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:31.009558 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ae81b-a6d5-4b40-a22f-449cceb77025 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:31.031492 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5201b30-fd21-4f95-9a10-94bf47a0ab8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:31.032146 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5201b30-fd21-4f95-9a10-94bf47a0ab8b admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:31.034057 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5201b30-fd21-4f95-9a10-94bf47a0ab8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:31.034904 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5201b30-fd21-4f95-9a10-94bf47a0ab8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:31.038191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5201b30-fd21-4f95-9a10-94bf47a0ab8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:31.044897 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5201b30-fd21-4f95-9a10-94bf47a0ab8b admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:31.086666 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:31.095892 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 882/1741] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:31.097542 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a10b60-96ef-4d73-9d3d-e092b85f4e82 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:31.118380 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a10b60-96ef-4d73-9d3d-e092b85f4e82 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:31.127953 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a9921-e4cc-4f24-8f5e-b6950df4e725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:31.127953 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a9921-e4cc-4f24-8f5e-b6950df4e725 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:31.135724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a9921-e4cc-4f24-8f5e-b6950df4e725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:31.136379 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a9921-e4cc-4f24-8f5e-b6950df4e725 admin admin] PATH_INFO: `/v3/projects/eb7f0f44d2164d25bc4f7f96cf1a39f9/users/65f5ec5cb07d4b41b569352058b02b28/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:31.184649 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75a9921-e4cc-4f24-8f5e-b6950df4e725 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb7f0f44d2164d25bc4f7f96cf1a39f9, user_id=65f5ec5cb07d4b41b569352058b02b28, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:31.186355 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75a9921-e4cc-4f24-8f5e-b6950df4e725 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:31.285599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 860/1742] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:25:31 2022] PUT /identity/v3/projects/eb7f0f44d2164d25bc4f7f96cf1a39f9/users/65f5ec5cb07d4b41b569352058b02b28/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:31.289432 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5201b30-fd21-4f95-9a10-94bf47a0ab8b admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:31.339253 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5201b30-fd21-4f95-9a10-94bf47a0ab8b admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:31.522054 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8adf17-dc35-4b9d-a802-16b212cbdf34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:31.524086 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8adf17-dc35-4b9d-a802-16b212cbdf34 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:31.526040 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8adf17-dc35-4b9d-a802-16b212cbdf34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:31.526513 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8adf17-dc35-4b9d-a802-16b212cbdf34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:31.527700 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8adf17-dc35-4b9d-a802-16b212cbdf34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:31.532574 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8adf17-dc35-4b9d-a802-16b212cbdf34 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:31.559317 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 883/1743] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:31.564222 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a9921-e4cc-4f24-8f5e-b6950df4e725 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:31.581302 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a9921-e4cc-4f24-8f5e-b6950df4e725 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:31.600901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af53e286-8b2a-4674-8040-e7ec1a53057b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:31.602244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53e286-8b2a-4674-8040-e7ec1a53057b admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:31.603204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53e286-8b2a-4674-8040-e7ec1a53057b admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:31.603657 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53e286-8b2a-4674-8040-e7ec1a53057b admin admin] PATH_INFO: `/v3/projects/eb7f0f44d2164d25bc4f7f96cf1a39f9/users/65f5ec5cb07d4b41b569352058b02b28/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:31.607474 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af53e286-8b2a-4674-8040-e7ec1a53057b admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb7f0f44d2164d25bc4f7f96cf1a39f9, user_id=65f5ec5cb07d4b41b569352058b02b28, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:31.635609 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af53e286-8b2a-4674-8040-e7ec1a53057b admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:31.655173 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 861/1744] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:25:31 2022] PUT /identity/v3/projects/eb7f0f44d2164d25bc4f7f96cf1a39f9/users/65f5ec5cb07d4b41b569352058b02b28/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:31.666477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0697a2-309a-4440-8db5-e2526139baac None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:31.666929 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0697a2-309a-4440-8db5-e2526139baac None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:31.667362 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0697a2-309a-4440-8db5-e2526139baac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:31.692621 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:31.743607 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-2c0697a2-309a-4440-8db5-e2526139baac None None] MFA Rules not processed for user `65f5ec5cb07d4b41b569352058b02b28`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:31.758230 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-2c0697a2-309a-4440-8db5-e2526139baac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:31.967269 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 884/1745] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:31 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:31.979822 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af53e286-8b2a-4674-8040-e7ec1a53057b admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:31.996523 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:32.089127 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:32.163146 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af53e286-8b2a-4674-8040-e7ec1a53057b admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:32.168138 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2851cf-568f-4b97-b178-7b54f4247c0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:32.169102 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2851cf-568f-4b97-b178-7b54f4247c0f service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:32.169548 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2851cf-568f-4b97-b178-7b54f4247c0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:32.183318 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2851cf-568f-4b97-b178-7b54f4247c0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:32.186442 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0697a2-309a-4440-8db5-e2526139baac None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:32.188476 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-2c0697a2-309a-4440-8db5-e2526139baac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:32.233689 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2851cf-568f-4b97-b178-7b54f4247c0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:32.235687 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2851cf-568f-4b97-b178-7b54f4247c0f service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:32.265980 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 862/1746] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:31 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:32.311062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2851cf-568f-4b97-b178-7b54f4247c0f service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:32.471613 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0697a2-309a-4440-8db5-e2526139baac None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:32.496943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b34a62e-58c9-4a69-a8e3-b6c4fc2ab911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:32.498339 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34a62e-58c9-4a69-a8e3-b6c4fc2ab911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:32.498618 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34a62e-58c9-4a69-a8e3-b6c4fc2ab911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:32.498894 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34a62e-58c9-4a69-a8e3-b6c4fc2ab911 admin admin] PATH_INFO: `/v3/projects/4de55490e2a84e4ba145a09dd6c9a877` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:32.506756 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b34a62e-58c9-4a69-a8e3-b6c4fc2ab911 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4de55490e2a84e4ba145a09dd6c9a877)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:32.509022 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b34a62e-58c9-4a69-a8e3-b6c4fc2ab911 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:32.540891 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:32.580776 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2851cf-568f-4b97-b178-7b54f4247c0f service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:32.603607 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc153e2f-d7d4-48ba-8b8c-659c6729b378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:32.607487 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc153e2f-d7d4-48ba-8b8c-659c6729b378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:32.607487 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc153e2f-d7d4-48ba-8b8c-659c6729b378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:32.607805 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc153e2f-d7d4-48ba-8b8c-659c6729b378 admin admin] PATH_INFO: `/v3/users/2d3c197b8f734f549589839633dc2359` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:32.610995 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc153e2f-d7d4-48ba-8b8c-659c6729b378 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d3c197b8f734f549589839633dc2359)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:32.614722 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc153e2f-d7d4-48ba-8b8c-659c6729b378 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:32.755824 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-5b34a62e-58c9-4a69-a8e3-b6c4fc2ab911 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4de55490e2a84e4ba145a09dd6c9a877', 'request_id': 'req-5b34a62e-58c9-4a69-a8e3-b6c4fc2ab911'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:32.778343 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-5b34a62e-58c9-4a69-a8e3-b6c4fc2ab911 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4de55490e2a84e4ba145a09dd6c9a877', 'request_id': 'req-5b34a62e-58c9-4a69-a8e3-b6c4fc2ab911'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:32.793567 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-5b34a62e-58c9-4a69-a8e3-b6c4fc2ab911 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4de55490e2a84e4ba145a09dd6c9a877', 'request_id': 'req-5b34a62e-58c9-4a69-a8e3-b6c4fc2ab911'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:32.889931 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-5b34a62e-58c9-4a69-a8e3-b6c4fc2ab911 admin admin] The token cache is being invalidate because project 4de55490e2a84e4ba145a09dd6c9a877 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:25:32.890385 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-5b34a62e-58c9-4a69-a8e3-b6c4fc2ab911 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:32.891640 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-5b34a62e-58c9-4a69-a8e3-b6c4fc2ab911 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:32.893931 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 885/1747] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:25:32 2022] DELETE /identity/v3/projects/4de55490e2a84e4ba145a09dd6c9a877 => generated 0 bytes in 708 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:32.899493 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278e22d-0ab1-4482-a4f5-5053589cf780 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:32.900806 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278e22d-0ab1-4482-a4f5-5053589cf780 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:32.901227 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278e22d-0ab1-4482-a4f5-5053589cf780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:32.909858 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-dc153e2f-d7d4-48ba-8b8c-659c6729b378 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d3c197b8f734f549589839633dc2359'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:32.943261 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-dc153e2f-d7d4-48ba-8b8c-659c6729b378 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d3c197b8f734f549589839633dc2359'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:33.016342 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-e278e22d-0ab1-4482-a4f5-5053589cf780 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:33.023918 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-e278e22d-0ab1-4482-a4f5-5053589cf780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:33.042000 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-dc153e2f-d7d4-48ba-8b8c-659c6729b378 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d3c197b8f734f549589839633dc2359'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:33.042681 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-dc153e2f-d7d4-48ba-8b8c-659c6729b378 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d3c197b8f734f549589839633dc2359'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:33.053492 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-dc153e2f-d7d4-48ba-8b8c-659c6729b378 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d3c197b8f734f549589839633dc2359'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:33.080227 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 863/1748] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:25:32 2022] DELETE /identity/v3/users/2d3c197b8f734f549589839633dc2359 => generated 0 bytes in 765 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:33.081893 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ebca6-f4f0-459d-bb2e-4a27d05cd44e None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:33.082082 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ebca6-f4f0-459d-bb2e-4a27d05cd44e None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:33.082395 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ebca6-f4f0-459d-bb2e-4a27d05cd44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:33.171120 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-967ebca6-f4f0-459d-bb2e-4a27d05cd44e None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:33.171120 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-967ebca6-f4f0-459d-bb2e-4a27d05cd44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:33.244165 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 886/1749] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:33.254041 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03116dcf-8251-4321-9fdd-1e06cea9861a None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:33.254256 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03116dcf-8251-4321-9fdd-1e06cea9861a None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:33.254740 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03116dcf-8251-4321-9fdd-1e06cea9861a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:33.293185 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-03116dcf-8251-4321-9fdd-1e06cea9861a None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:33.294640 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-03116dcf-8251-4321-9fdd-1e06cea9861a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:33.363044 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:33.367877 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 864/1750] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:33.409034 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 887/1751] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:33.414600 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ebca6-f4f0-459d-bb2e-4a27d05cd44e None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:33.430745 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ebca6-f4f0-459d-bb2e-4a27d05cd44e None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:33.440132 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e0da50-ba3e-4adc-bdcb-3fca02f7c190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:33.441193 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0da50-ba3e-4adc-bdcb-3fca02f7c190 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:33.441748 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0da50-ba3e-4adc-bdcb-3fca02f7c190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:33.442499 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0da50-ba3e-4adc-bdcb-3fca02f7c190 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:33.443874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e0da50-ba3e-4adc-bdcb-3fca02f7c190 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:25:33.444583 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e0da50-ba3e-4adc-bdcb-3fca02f7c190 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:33.462947 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e0da50-ba3e-4adc-bdcb-3fca02f7c190 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:33.471690 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03116dcf-8251-4321-9fdd-1e06cea9861a None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:33.473794 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-03116dcf-8251-4321-9fdd-1e06cea9861a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:33.481563 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 865/1752] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:25:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:25:33.485202 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e0da50-ba3e-4adc-bdcb-3fca02f7c190 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:33.496872 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e0da50-ba3e-4adc-bdcb-3fca02f7c190 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:33.503203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc79254-5b4e-494f-90be-d768ad5999c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:33.504310 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc79254-5b4e-494f-90be-d768ad5999c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:33.504975 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc79254-5b4e-494f-90be-d768ad5999c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:33.505685 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc79254-5b4e-494f-90be-d768ad5999c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:33.506569 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc79254-5b4e-494f-90be-d768ad5999c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:33.509003 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc79254-5b4e-494f-90be-d768ad5999c2 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:33.562111 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:33.589339 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03116dcf-8251-4321-9fdd-1e06cea9861a None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:33.596776 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64721087-ea9e-49e3-926e-4aa1c59eec2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:33.599636 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64721087-ea9e-49e3-926e-4aa1c59eec2b service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:33.601286 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64721087-ea9e-49e3-926e-4aa1c59eec2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:33.601766 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64721087-ea9e-49e3-926e-4aa1c59eec2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:33.617150 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 866/1753] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:25:33 2022] POST /identity/v3/projects => generated 367 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:25:33.622039 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc79254-5b4e-494f-90be-d768ad5999c2 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:33.640447 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64721087-ea9e-49e3-926e-4aa1c59eec2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:33.644202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64721087-ea9e-49e3-926e-4aa1c59eec2b service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:33.669841 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 888/1754] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:33.732717 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc79254-5b4e-494f-90be-d768ad5999c2 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:33.740758 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf1d90-d211-4160-ac09-e3544ab70a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:33.742538 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf1d90-d211-4160-ac09-e3544ab70a4d admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:33.742538 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf1d90-d211-4160-ac09-e3544ab70a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:33.742538 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf1d90-d211-4160-ac09-e3544ab70a4d admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:33.743202 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badf1d90-d211-4160-ac09-e3544ab70a4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:33.745763 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badf1d90-d211-4160-ac09-e3544ab70a4d admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:33.863970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 867/1755] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:25:33 2022] POST /identity/v3/users => generated 284 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:33.869228 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64721087-ea9e-49e3-926e-4aa1c59eec2b service neutron] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:33.884358 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64721087-ea9e-49e3-926e-4aa1c59eec2b service neutron] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:33.891550 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40ddbfd-96ef-4885-8c9f-7efb84f92def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:33.892513 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ddbfd-96ef-4885-8c9f-7efb84f92def admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:33.893116 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ddbfd-96ef-4885-8c9f-7efb84f92def admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:33.893689 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ddbfd-96ef-4885-8c9f-7efb84f92def admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:33.895691 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40ddbfd-96ef-4885-8c9f-7efb84f92def admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:33.898199 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40ddbfd-96ef-4885-8c9f-7efb84f92def admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:33.914871 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 889/1756] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:25:33.918674 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf1d90-d211-4160-ac09-e3544ab70a4d admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:33.935758 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf1d90-d211-4160-ac09-e3544ab70a4d admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:33.956021 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02c01c2-d29b-46ba-8f83-79bc0890a9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:33.956260 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c01c2-d29b-46ba-8f83-79bc0890a9d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:33.956487 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c01c2-d29b-46ba-8f83-79bc0890a9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:33.956777 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c01c2-d29b-46ba-8f83-79bc0890a9d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:33.957437 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02c01c2-d29b-46ba-8f83-79bc0890a9d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:33.961163 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02c01c2-d29b-46ba-8f83-79bc0890a9d6 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:34.004987 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40ddbfd-96ef-4885-8c9f-7efb84f92def admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:34.005178 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 868/1757] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:34.036324 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40ddbfd-96ef-4885-8c9f-7efb84f92def admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:34.059532 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b5e92-d2fe-4a1e-8edb-381e55bed97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:34.060791 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b5e92-d2fe-4a1e-8edb-381e55bed97c admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:34.061306 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b5e92-d2fe-4a1e-8edb-381e55bed97c admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:34.061731 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b5e92-d2fe-4a1e-8edb-381e55bed97c admin admin] PATH_INFO: `/v3/projects/587c3fc6a8f747509af8f09bfbabd673/users/9475bab03724429b8273fc9e3c1bc7df/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:34.113594 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b5e92-d2fe-4a1e-8edb-381e55bed97c admin admin] RBAC: Authorizing `identity:create_grant(project_id=587c3fc6a8f747509af8f09bfbabd673, user_id=9475bab03724429b8273fc9e3c1bc7df, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:34.116153 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b5e92-d2fe-4a1e-8edb-381e55bed97c admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:34.203131 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 890/1758] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:25:34 2022] PUT /identity/v3/projects/587c3fc6a8f747509af8f09bfbabd673/users/9475bab03724429b8273fc9e3c1bc7df/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:25:34.214926 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5184b-e773-4db0-83c8-b66579d3d892 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:34.215516 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5184b-e773-4db0-83c8-b66579d3d892 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:34.216041 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5184b-e773-4db0-83c8-b66579d3d892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:34.320880 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-53b5184b-e773-4db0-83c8-b66579d3d892 None None] MFA Rules not processed for user `9475bab03724429b8273fc9e3c1bc7df`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:34.324021 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-53b5184b-e773-4db0-83c8-b66579d3d892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:34.394751 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:34.602828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 869/1759] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 389 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:25:35.329159 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b5e92-d2fe-4a1e-8edb-381e55bed97c admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:35.331146 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-a07b5e92-d2fe-4a1e-8edb-381e55bed97c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:35.460733 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:35.494849 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b5e92-d2fe-4a1e-8edb-381e55bed97c admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:35.508234 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e3ca4-ef16-4d40-b94c-69ad0bddd752 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:35.508994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e3ca4-ef16-4d40-b94c-69ad0bddd752 service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:35.509278 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e3ca4-ef16-4d40-b94c-69ad0bddd752 service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:35.509565 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e3ca4-ef16-4d40-b94c-69ad0bddd752 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:35.536165 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e3ca4-ef16-4d40-b94c-69ad0bddd752 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:35.538220 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e3ca4-ef16-4d40-b94c-69ad0bddd752 service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:35.587228 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 891/1760] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:36.037168 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ce10d-387a-4889-b04d-d0b314c53f7d None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:36.038051 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ce10d-387a-4889-b04d-d0b314c53f7d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:36.039909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ce10d-387a-4889-b04d-d0b314c53f7d None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:36.040067 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 870/1761] 10.222.0.26 () {54 vars in 921 bytes} [Fri Aug 5 14:25:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:25:36.061785 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8cd6b8-2220-4f2b-a307-1ac11294c348 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:36.062259 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8cd6b8-2220-4f2b-a307-1ac11294c348 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:36.062678 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8cd6b8-2220-4f2b-a307-1ac11294c348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:36.130401 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-9b8cd6b8-2220-4f2b-a307-1ac11294c348 None None] MFA Rules not processed for user `5ec4940c99fc4fa8a5bc7cdabc4558fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:36.131882 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-9b8cd6b8-2220-4f2b-a307-1ac11294c348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:36.245809 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 892/1762] 10.222.0.26 () {58 vars in 1018 bytes} [Fri Aug 5 14:25:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:36.319740 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ce10d-387a-4889-b04d-d0b314c53f7d None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:36.324853 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d50ce10d-387a-4889-b04d-d0b314c53f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:36.363190 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ce10d-387a-4889-b04d-d0b314c53f7d None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:36.378051 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57339b6-41e4-4179-814e-9ddd937d7c09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:36.378694 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57339b6-41e4-4179-814e-9ddd937d7c09 service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:36.379159 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57339b6-41e4-4179-814e-9ddd937d7c09 service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:36.379578 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57339b6-41e4-4179-814e-9ddd937d7c09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:36.393729 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57339b6-41e4-4179-814e-9ddd937d7c09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:36.396478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57339b6-41e4-4179-814e-9ddd937d7c09 service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:36.410344 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 871/1763] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:36.606403 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa436f-680d-4923-aa38-7cc9ece5ba3a None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:36.606989 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa436f-680d-4923-aa38-7cc9ece5ba3a None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:36.606989 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa436f-680d-4923-aa38-7cc9ece5ba3a None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:36.608294 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 893/1764] 10.222.0.26 () {54 vars in 921 bytes} [Fri Aug 5 14:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:25:36.633613 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f16144-a705-4f22-a8d0-82982c6e7bd2 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:36.633613 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f16144-a705-4f22-a8d0-82982c6e7bd2 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:36.633613 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f16144-a705-4f22-a8d0-82982c6e7bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:36.642787 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:36.673930 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-22f16144-a705-4f22-a8d0-82982c6e7bd2 None None] MFA Rules not processed for user `5ec4940c99fc4fa8a5bc7cdabc4558fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:36.679561 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-22f16144-a705-4f22-a8d0-82982c6e7bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:36.722164 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 872/1765] 10.222.0.26 () {58 vars in 1018 bytes} [Fri Aug 5 14:25:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:36.779564 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa436f-680d-4923-aa38-7cc9ece5ba3a None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:36.799471 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa436f-680d-4923-aa38-7cc9ece5ba3a None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:36.809088 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a4ef0-4f4a-48fd-8a55-f6c202f27b11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:36.809088 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a4ef0-4f4a-48fd-8a55-f6c202f27b11 service nova] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:36.809088 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a4ef0-4f4a-48fd-8a55-f6c202f27b11 service nova] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:36.809088 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a4ef0-4f4a-48fd-8a55-f6c202f27b11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:36.840006 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5a4ef0-4f4a-48fd-8a55-f6c202f27b11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:36.849100 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5a4ef0-4f4a-48fd-8a55-f6c202f27b11 service nova] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:36.879369 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 894/1766] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:36.900101 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfd8a3-417e-4933-97c7-81dc839694b0 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:36.901118 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfd8a3-417e-4933-97c7-81dc839694b0 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:36.901118 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfd8a3-417e-4933-97c7-81dc839694b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:36.927094 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-e6bfd8a3-417e-4933-97c7-81dc839694b0 None None] MFA Rules not processed for user `99dc94fff06f42c39f3750484f6d8e9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:36.928732 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e6bfd8a3-417e-4933-97c7-81dc839694b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:37.036742 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 873/1767] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:37.068928 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a4ef0-4f4a-48fd-8a55-f6c202f27b11 service nova] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:37.070452 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-5a5a4ef0-4f4a-48fd-8a55-f6c202f27b11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:37.212135 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a4ef0-4f4a-48fd-8a55-f6c202f27b11 service nova] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:37.218475 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6e85c-39e0-4856-9dc9-ad1e6603a222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:37.219536 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6e85c-39e0-4856-9dc9-ad1e6603a222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:37.220064 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6e85c-39e0-4856-9dc9-ad1e6603a222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:37.220550 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6e85c-39e0-4856-9dc9-ad1e6603a222 admin admin] PATH_INFO: `/v3/projects/c1996206308c4dc68cee8ef3ed4a5dfb` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:37.223407 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d6e85c-39e0-4856-9dc9-ad1e6603a222 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1996206308c4dc68cee8ef3ed4a5dfb)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:37.225452 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d6e85c-39e0-4856-9dc9-ad1e6603a222 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:37.292081 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:37.369038 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-88d6e85c-39e0-4856-9dc9-ad1e6603a222 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1996206308c4dc68cee8ef3ed4a5dfb', 'request_id': 'req-88d6e85c-39e0-4856-9dc9-ad1e6603a222'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:37.374985 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-88d6e85c-39e0-4856-9dc9-ad1e6603a222 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1996206308c4dc68cee8ef3ed4a5dfb', 'request_id': 'req-88d6e85c-39e0-4856-9dc9-ad1e6603a222'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:37.381161 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-88d6e85c-39e0-4856-9dc9-ad1e6603a222 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1996206308c4dc68cee8ef3ed4a5dfb', 'request_id': 'req-88d6e85c-39e0-4856-9dc9-ad1e6603a222'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:37.452954 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-88d6e85c-39e0-4856-9dc9-ad1e6603a222 admin admin] The token cache is being invalidate because project c1996206308c4dc68cee8ef3ed4a5dfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:25:37.453319 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-88d6e85c-39e0-4856-9dc9-ad1e6603a222 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:37.453962 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-88d6e85c-39e0-4856-9dc9-ad1e6603a222 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:37.455495 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 895/1768] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:25:37 2022] DELETE /identity/v3/projects/c1996206308c4dc68cee8ef3ed4a5dfb => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:37.459616 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bfd8a3-417e-4933-97c7-81dc839694b0 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:37.470579 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bfd8a3-417e-4933-97c7-81dc839694b0 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:37.532883 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc4d42-ae0f-4d23-95d0-9ddc37374107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:37.533533 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc4d42-ae0f-4d23-95d0-9ddc37374107 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:37.533810 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc4d42-ae0f-4d23-95d0-9ddc37374107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:37.534079 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc4d42-ae0f-4d23-95d0-9ddc37374107 admin admin] PATH_INFO: `/v3/users/65f5ec5cb07d4b41b569352058b02b28` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:37.535240 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fc4d42-ae0f-4d23-95d0-9ddc37374107 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65f5ec5cb07d4b41b569352058b02b28)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:37.538639 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fc4d42-ae0f-4d23-95d0-9ddc37374107 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:37.678348 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d405b9-5632-4b73-a53c-80ad6535ed73 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:37.678822 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d405b9-5632-4b73-a53c-80ad6535ed73 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:37.678910 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d405b9-5632-4b73-a53c-80ad6535ed73 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:37.680267 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 896/1769] 10.222.0.26 () {54 vars in 912 bytes} [Fri Aug 5 14:25:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:25:37.686433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa8a35-14ae-4d94-bca1-3d834c606ca7 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:37.686721 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa8a35-14ae-4d94-bca1-3d834c606ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:37.686999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa8a35-14ae-4d94-bca1-3d834c606ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:37.707128 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-95fa8a35-14ae-4d94-bca1-3d834c606ca7 None None] MFA Rules not processed for user `695e37560f8f495e987d1bd17acc2680`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:37.709223 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-95fa8a35-14ae-4d94-bca1-3d834c606ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:37.763498 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 897/1770] 10.222.0.26 () {58 vars in 1009 bytes} [Fri Aug 5 14:25:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:25:37.771252 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-56fc4d42-ae0f-4d23-95d0-9ddc37374107 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65f5ec5cb07d4b41b569352058b02b28'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:37.791390 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-56fc4d42-ae0f-4d23-95d0-9ddc37374107 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65f5ec5cb07d4b41b569352058b02b28'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:37.815582 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa8a35-14ae-4d94-bca1-3d834c606ca7 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:37.822913 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-95fa8a35-14ae-4d94-bca1-3d834c606ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:37.843305 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa8a35-14ae-4d94-bca1-3d834c606ca7 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:37.847776 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8157d7-7614-4b6f-8d4c-8e512bea8ef0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:37.849228 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8157d7-7614-4b6f-8d4c-8e512bea8ef0 service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:37.850292 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8157d7-7614-4b6f-8d4c-8e512bea8ef0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:37.850292 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8157d7-7614-4b6f-8d4c-8e512bea8ef0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:37.859803 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8157d7-7614-4b6f-8d4c-8e512bea8ef0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:37.861694 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8157d7-7614-4b6f-8d4c-8e512bea8ef0 service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:37.875775 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 898/1771] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 14:25:37.898862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-56fc4d42-ae0f-4d23-95d0-9ddc37374107 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65f5ec5cb07d4b41b569352058b02b28'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:37.902797 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-56fc4d42-ae0f-4d23-95d0-9ddc37374107 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65f5ec5cb07d4b41b569352058b02b28'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:37.917590 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-56fc4d42-ae0f-4d23-95d0-9ddc37374107 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65f5ec5cb07d4b41b569352058b02b28'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:37.917739 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e301e0-13aa-4d2b-bd68-7749f253e6e7 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:37.918261 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e301e0-13aa-4d2b-bd68-7749f253e6e7 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:37.918546 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e301e0-13aa-4d2b-bd68-7749f253e6e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:37.923391 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 874/1772] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:25:37 2022] DELETE /identity/v3/users/65f5ec5cb07d4b41b569352058b02b28 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:37.943859 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-05e301e0-13aa-4d2b-bd68-7749f253e6e7 None None] MFA Rules not processed for user `084b18670c2a4c1396d5912d130b2bac`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:37.946181 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-05e301e0-13aa-4d2b-bd68-7749f253e6e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:38.032808 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 899/1773] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:37 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:25:38.426116 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc4d42-ae0f-4d23-95d0-9ddc37374107 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:38.429669 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-56fc4d42-ae0f-4d23-95d0-9ddc37374107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:38.472326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:38.498942 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:38.547803 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc4d42-ae0f-4d23-95d0-9ddc37374107 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:38.557357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91edaad-17a5-4ea8-a905-aaf02fe0ae93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:38.558117 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91edaad-17a5-4ea8-a905-aaf02fe0ae93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:38.558467 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91edaad-17a5-4ea8-a905-aaf02fe0ae93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:38.558946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91edaad-17a5-4ea8-a905-aaf02fe0ae93 admin admin] PATH_INFO: `/v3/projects/eb7f0f44d2164d25bc4f7f96cf1a39f9` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:38.560284 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91edaad-17a5-4ea8-a905-aaf02fe0ae93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb7f0f44d2164d25bc4f7f96cf1a39f9)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:38.562293 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91edaad-17a5-4ea8-a905-aaf02fe0ae93 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:38.682500 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c91edaad-17a5-4ea8-a905-aaf02fe0ae93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb7f0f44d2164d25bc4f7f96cf1a39f9', 'request_id': 'req-c91edaad-17a5-4ea8-a905-aaf02fe0ae93'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:38.693411 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c91edaad-17a5-4ea8-a905-aaf02fe0ae93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb7f0f44d2164d25bc4f7f96cf1a39f9', 'request_id': 'req-c91edaad-17a5-4ea8-a905-aaf02fe0ae93'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:38.788128 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c91edaad-17a5-4ea8-a905-aaf02fe0ae93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb7f0f44d2164d25bc4f7f96cf1a39f9', 'request_id': 'req-c91edaad-17a5-4ea8-a905-aaf02fe0ae93'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:38.799981 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c91edaad-17a5-4ea8-a905-aaf02fe0ae93 admin admin] The token cache is being invalidate because project eb7f0f44d2164d25bc4f7f96cf1a39f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:25:38.800520 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c91edaad-17a5-4ea8-a905-aaf02fe0ae93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:38.801198 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c91edaad-17a5-4ea8-a905-aaf02fe0ae93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:38.803081 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 875/1774] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:25:38 2022] DELETE /identity/v3/projects/eb7f0f44d2164d25bc4f7f96cf1a39f9 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:38.808133 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eccb21-4483-4e75-aa70-cdc6ca43c1d8 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:38.808685 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eccb21-4483-4e75-aa70-cdc6ca43c1d8 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:38.808779 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eccb21-4483-4e75-aa70-cdc6ca43c1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:38.830560 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-32eccb21-4483-4e75-aa70-cdc6ca43c1d8 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:38.832731 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-32eccb21-4483-4e75-aa70-cdc6ca43c1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:38.914461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 900/1775] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:25:38.924940 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc3e0e-6fb1-4795-b08b-37d609a3667b None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:38.925289 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc3e0e-6fb1-4795-b08b-37d609a3667b None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:38.926597 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc3e0e-6fb1-4795-b08b-37d609a3667b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:38.958525 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-f7dc3e0e-6fb1-4795-b08b-37d609a3667b None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:38.960006 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f7dc3e0e-6fb1-4795-b08b-37d609a3667b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:38.968952 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 876/1776] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:38.973846 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eccb21-4483-4e75-aa70-cdc6ca43c1d8 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:38.985436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eccb21-4483-4e75-aa70-cdc6ca43c1d8 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:38.991289 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aeefc4-af22-43bc-8b83-770a4346f69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:38.991919 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aeefc4-af22-43bc-8b83-770a4346f69e admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:38.993518 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aeefc4-af22-43bc-8b83-770a4346f69e admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:38.993518 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aeefc4-af22-43bc-8b83-770a4346f69e admin admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:38.993518 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aeefc4-af22-43bc-8b83-770a4346f69e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:25:38.993518 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aeefc4-af22-43bc-8b83-770a4346f69e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:38.995335 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aeefc4-af22-43bc-8b83-770a4346f69e admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:39.012089 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 901/1777] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:25:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:25:39.016649 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc3e0e-6fb1-4795-b08b-37d609a3667b None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:39.026865 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc3e0e-6fb1-4795-b08b-37d609a3667b None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:39.031759 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd42a4e-a776-4d5b-95ca-33c37bb8f18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:39.032378 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd42a4e-a776-4d5b-95ca-33c37bb8f18b admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:39.032690 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd42a4e-a776-4d5b-95ca-33c37bb8f18b admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:39.032970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd42a4e-a776-4d5b-95ca-33c37bb8f18b admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:39.033629 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd42a4e-a776-4d5b-95ca-33c37bb8f18b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:39.036059 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd42a4e-a776-4d5b-95ca-33c37bb8f18b admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:39.153388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 877/1778] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:25:39 2022] POST /identity/v3/projects => generated 391 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:39.163793 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aeefc4-af22-43bc-8b83-770a4346f69e admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:39.174809 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aeefc4-af22-43bc-8b83-770a4346f69e admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:39.243259 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053d699-f657-4e40-ae7c-40545b1f1613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:39.243259 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053d699-f657-4e40-ae7c-40545b1f1613 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:39.243259 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053d699-f657-4e40-ae7c-40545b1f1613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:39.243545 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053d699-f657-4e40-ae7c-40545b1f1613 admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:39.244301 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2053d699-f657-4e40-ae7c-40545b1f1613 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:39.246258 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2053d699-f657-4e40-ae7c-40545b1f1613 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:39.377242 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 902/1779] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:25:39 2022] POST /identity/v3/users => generated 296 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:39.384551 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd42a4e-a776-4d5b-95ca-33c37bb8f18b admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:39.395849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd42a4e-a776-4d5b-95ca-33c37bb8f18b admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:39.400465 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5969dd1-ba6a-49f6-9514-1763c75cca77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:39.401074 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5969dd1-ba6a-49f6-9514-1763c75cca77 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:39.401351 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5969dd1-ba6a-49f6-9514-1763c75cca77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:39.401620 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5969dd1-ba6a-49f6-9514-1763c75cca77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:39.402307 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5969dd1-ba6a-49f6-9514-1763c75cca77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:39.403914 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5969dd1-ba6a-49f6-9514-1763c75cca77 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:39.414402 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 878/1780] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:39.418562 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053d699-f657-4e40-ae7c-40545b1f1613 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:39.429210 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053d699-f657-4e40-ae7c-40545b1f1613 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:39.433906 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9b8011-dbec-4be6-9a41-34df1d1a7941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:39.434504 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b8011-dbec-4be6-9a41-34df1d1a7941 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:39.434782 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b8011-dbec-4be6-9a41-34df1d1a7941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:39.435051 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b8011-dbec-4be6-9a41-34df1d1a7941 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:39.435734 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9b8011-dbec-4be6-9a41-34df1d1a7941 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:39.437346 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9b8011-dbec-4be6-9a41-34df1d1a7941 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:39.448427 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 903/1781] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:39.452437 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5969dd1-ba6a-49f6-9514-1763c75cca77 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:39.462880 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5969dd1-ba6a-49f6-9514-1763c75cca77 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:39.467642 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403676a0-3fec-4ec0-9833-d356d20fb4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:39.468611 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403676a0-3fec-4ec0-9833-d356d20fb4b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:39.468891 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403676a0-3fec-4ec0-9833-d356d20fb4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:39.469158 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403676a0-3fec-4ec0-9833-d356d20fb4b3 admin admin] PATH_INFO: `/v3/projects/0cde47c8423348408353e74a792d98b1/users/eb6de716bfac4c8d8c7be2b7fe274778/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:39.484980 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403676a0-3fec-4ec0-9833-d356d20fb4b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cde47c8423348408353e74a792d98b1, user_id=eb6de716bfac4c8d8c7be2b7fe274778, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:39.487673 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403676a0-3fec-4ec0-9833-d356d20fb4b3 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:39.586373 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 879/1782] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:25:39 2022] PUT /identity/v3/projects/0cde47c8423348408353e74a792d98b1/users/eb6de716bfac4c8d8c7be2b7fe274778/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:39.598987 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84fef94-fe75-4c26-a72b-b199918ecf2f None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:39.598987 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84fef94-fe75-4c26-a72b-b199918ecf2f None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:39.598987 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84fef94-fe75-4c26-a72b-b199918ecf2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:39.628003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-a84fef94-fe75-4c26-a72b-b199918ecf2f None None] MFA Rules not processed for user `eb6de716bfac4c8d8c7be2b7fe274778`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:39.629281 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-a84fef94-fe75-4c26-a72b-b199918ecf2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:39.723897 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403676a0-3fec-4ec0-9833-d356d20fb4b3 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:39.730679 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 904/1783] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:39 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:39.835291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403676a0-3fec-4ec0-9833-d356d20fb4b3 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:39.842302 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78db27-7b10-4343-9783-d2b3b5fedab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:39.843707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78db27-7b10-4343-9783-d2b3b5fedab3 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:39.844228 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78db27-7b10-4343-9783-d2b3b5fedab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:39.845123 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78db27-7b10-4343-9783-d2b3b5fedab3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:39.846275 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c78db27-7b10-4343-9783-d2b3b5fedab3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:39.850152 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c78db27-7b10-4343-9783-d2b3b5fedab3 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:40.008751 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 880/1784] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:25:39 2022] POST /identity/v3/projects => generated 393 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:25:40.016004 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84fef94-fe75-4c26-a72b-b199918ecf2f None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:40.048586 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84fef94-fe75-4c26-a72b-b199918ecf2f None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:40.214416 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4227c2b-61e8-4abe-80a2-c46c87a6b891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:40.215055 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4227c2b-61e8-4abe-80a2-c46c87a6b891 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:40.215376 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4227c2b-61e8-4abe-80a2-c46c87a6b891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:40.215654 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4227c2b-61e8-4abe-80a2-c46c87a6b891 admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:40.216322 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4227c2b-61e8-4abe-80a2-c46c87a6b891 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:40.218665 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4227c2b-61e8-4abe-80a2-c46c87a6b891 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:40.331442 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 905/1785] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:25:40 2022] POST /identity/v3/users => generated 303 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:40.335891 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78db27-7b10-4343-9783-d2b3b5fedab3 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:40.350826 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78db27-7b10-4343-9783-d2b3b5fedab3 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:40.355995 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d1a20-4a08-4e11-89cd-5bb9458927c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:40.356905 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d1a20-4a08-4e11-89cd-5bb9458927c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:40.357180 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d1a20-4a08-4e11-89cd-5bb9458927c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:40.357450 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d1a20-4a08-4e11-89cd-5bb9458927c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:40.358148 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17d1a20-4a08-4e11-89cd-5bb9458927c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:40.360201 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17d1a20-4a08-4e11-89cd-5bb9458927c7 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:40.376352 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 881/1786] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:40.387994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4227c2b-61e8-4abe-80a2-c46c87a6b891 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:40.399953 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4227c2b-61e8-4abe-80a2-c46c87a6b891 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:40.423378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1218fba-3dda-413e-978d-32d0cc720e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:40.423378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1218fba-3dda-413e-978d-32d0cc720e4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:40.423378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1218fba-3dda-413e-978d-32d0cc720e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:40.423378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1218fba-3dda-413e-978d-32d0cc720e4e admin admin] PATH_INFO: `/v3/projects/3fa111a0a0fc48459268a62f8b48915f/users/9d041dfc595f42caacb4066221c07d95/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:40.445088 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1218fba-3dda-413e-978d-32d0cc720e4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fa111a0a0fc48459268a62f8b48915f, user_id=9d041dfc595f42caacb4066221c07d95, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:40.448034 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1218fba-3dda-413e-978d-32d0cc720e4e admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:40.534166 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 906/1787] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:25:40 2022] PUT /identity/v3/projects/3fa111a0a0fc48459268a62f8b48915f/users/9d041dfc595f42caacb4066221c07d95/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:40.539580 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d1a20-4a08-4e11-89cd-5bb9458927c7 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:40.553578 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d1a20-4a08-4e11-89cd-5bb9458927c7 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:40.627664 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e11fae4-0f09-4555-96e3-01b8f7cd4ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:40.628679 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11fae4-0f09-4555-96e3-01b8f7cd4ab6 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:40.629893 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11fae4-0f09-4555-96e3-01b8f7cd4ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:40.630462 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11fae4-0f09-4555-96e3-01b8f7cd4ab6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:40.631393 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e11fae4-0f09-4555-96e3-01b8f7cd4ab6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:40.633194 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e11fae4-0f09-4555-96e3-01b8f7cd4ab6 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:40.649072 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 882/1788] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:40.649780 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1218fba-3dda-413e-978d-32d0cc720e4e admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:40.661572 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1218fba-3dda-413e-978d-32d0cc720e4e admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:40.683362 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5d2db-55b9-42ee-87f1-bb3d14dd8fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:40.683362 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5d2db-55b9-42ee-87f1-bb3d14dd8fce admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:40.683362 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5d2db-55b9-42ee-87f1-bb3d14dd8fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:40.683362 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5d2db-55b9-42ee-87f1-bb3d14dd8fce admin admin] PATH_INFO: `/v3/projects/3fa111a0a0fc48459268a62f8b48915f/users/9d041dfc595f42caacb4066221c07d95/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:40.690077 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a5d2db-55b9-42ee-87f1-bb3d14dd8fce admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fa111a0a0fc48459268a62f8b48915f, user_id=9d041dfc595f42caacb4066221c07d95, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:40.694760 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a5d2db-55b9-42ee-87f1-bb3d14dd8fce admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:40.710768 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 907/1789] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:25:40 2022] PUT /identity/v3/projects/3fa111a0a0fc48459268a62f8b48915f/users/9d041dfc595f42caacb4066221c07d95/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:40.721628 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8e3f9-777b-4f9d-8823-1974b87b4e62 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:40.722178 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8e3f9-777b-4f9d-8823-1974b87b4e62 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:40.725602 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8e3f9-777b-4f9d-8823-1974b87b4e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:40.727063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:40.753555 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-77e8e3f9-777b-4f9d-8823-1974b87b4e62 None None] MFA Rules not processed for user `9d041dfc595f42caacb4066221c07d95`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:40.755186 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-77e8e3f9-777b-4f9d-8823-1974b87b4e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:40.836994 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 883/1790] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:40 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:40.850204 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5d2db-55b9-42ee-87f1-bb3d14dd8fce admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:40.851666 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-b7a5d2db-55b9-42ee-87f1-bb3d14dd8fce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:40.872469 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:40.924002 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5d2db-55b9-42ee-87f1-bb3d14dd8fce admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:40.929195 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd257b4-2cd1-45ff-a126-b75d50783d43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:40.929829 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd257b4-2cd1-45ff-a126-b75d50783d43 service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:40.930106 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd257b4-2cd1-45ff-a126-b75d50783d43 service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:40.930383 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd257b4-2cd1-45ff-a126-b75d50783d43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:40.941159 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd257b4-2cd1-45ff-a126-b75d50783d43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:40.943185 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd257b4-2cd1-45ff-a126-b75d50783d43 service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:40.953582 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 908/1791] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:40 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 14:25:41.027098 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e8e3f9-777b-4f9d-8823-1974b87b4e62 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:41.038302 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e8e3f9-777b-4f9d-8823-1974b87b4e62 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:41.042392 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f62e1-f516-4963-8633-902f3e451d11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:41.043119 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f62e1-f516-4963-8633-902f3e451d11 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:41.043493 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f62e1-f516-4963-8633-902f3e451d11 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:41.043863 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f62e1-f516-4963-8633-902f3e451d11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:41.109441 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0f62e1-f516-4963-8633-902f3e451d11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:41.110907 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0f62e1-f516-4963-8633-902f3e451d11 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:41.119889 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 884/1792] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:41 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:41.390436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:41.710768 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:47.657387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd257b4-2cd1-45ff-a126-b75d50783d43 service cinder] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:47.766976 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd257b4-2cd1-45ff-a126-b75d50783d43 service cinder] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:47.772053 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b57fc-efef-49a6-ad26-44a2dc7da1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:47.772906 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b57fc-efef-49a6-ad26-44a2dc7da1fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:47.773238 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b57fc-efef-49a6-ad26-44a2dc7da1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:47.773519 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b57fc-efef-49a6-ad26-44a2dc7da1fe admin admin] PATH_INFO: `/v3/users/eb6de716bfac4c8d8c7be2b7fe274778` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:47.775173 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96b57fc-efef-49a6-ad26-44a2dc7da1fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb6de716bfac4c8d8c7be2b7fe274778)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:47.777424 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96b57fc-efef-49a6-ad26-44a2dc7da1fe admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:48.105901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-b96b57fc-efef-49a6-ad26-44a2dc7da1fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb6de716bfac4c8d8c7be2b7fe274778'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:48.132279 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-b96b57fc-efef-49a6-ad26-44a2dc7da1fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb6de716bfac4c8d8c7be2b7fe274778'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:48.133386 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-b96b57fc-efef-49a6-ad26-44a2dc7da1fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb6de716bfac4c8d8c7be2b7fe274778'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:48.134596 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-b96b57fc-efef-49a6-ad26-44a2dc7da1fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb6de716bfac4c8d8c7be2b7fe274778'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:48.160902 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-b96b57fc-efef-49a6-ad26-44a2dc7da1fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb6de716bfac4c8d8c7be2b7fe274778'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:48.238489 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 909/1793] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:25:47 2022] DELETE /identity/v3/users/eb6de716bfac4c8d8c7be2b7fe274778 => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:48.242367 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d00c31-492a-4809-b55c-35dfb84ae9e1 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:48.242903 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d00c31-492a-4809-b55c-35dfb84ae9e1 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:48.242903 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d00c31-492a-4809-b55c-35dfb84ae9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:48.270943 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-15d00c31-492a-4809-b55c-35dfb84ae9e1 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:48.272468 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-15d00c31-492a-4809-b55c-35dfb84ae9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:48.341475 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 885/1794] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:48.366549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b57fc-efef-49a6-ad26-44a2dc7da1fe admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:48.368595 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-b96b57fc-efef-49a6-ad26-44a2dc7da1fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:48.435786 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:48.460318 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b57fc-efef-49a6-ad26-44a2dc7da1fe admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:48.465623 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e9c11-d86a-401c-b531-1922f0ffca48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:48.466856 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e9c11-d86a-401c-b531-1922f0ffca48 service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:48.473866 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e9c11-d86a-401c-b531-1922f0ffca48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:48.473866 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e9c11-d86a-401c-b531-1922f0ffca48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:48.487368 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102e9c11-d86a-401c-b531-1922f0ffca48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:48.489183 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102e9c11-d86a-401c-b531-1922f0ffca48 service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:48.504468 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 910/1795] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:48.780067 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d00c31-492a-4809-b55c-35dfb84ae9e1 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:48.781787 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-15d00c31-492a-4809-b55c-35dfb84ae9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:48.800526 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d00c31-492a-4809-b55c-35dfb84ae9e1 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:48.805484 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58779d2f-8a6e-4e58-90b8-5d4e59977dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:48.806140 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58779d2f-8a6e-4e58-90b8-5d4e59977dd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:48.806453 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58779d2f-8a6e-4e58-90b8-5d4e59977dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:48.806731 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58779d2f-8a6e-4e58-90b8-5d4e59977dd3 admin admin] PATH_INFO: `/v3/users/0a25fd7b402e41068a7353de62e76d8c` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:48.808091 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58779d2f-8a6e-4e58-90b8-5d4e59977dd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a25fd7b402e41068a7353de62e76d8c)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:48.810097 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58779d2f-8a6e-4e58-90b8-5d4e59977dd3 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:49.042129 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-58779d2f-8a6e-4e58-90b8-5d4e59977dd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a25fd7b402e41068a7353de62e76d8c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:49.057697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-58779d2f-8a6e-4e58-90b8-5d4e59977dd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a25fd7b402e41068a7353de62e76d8c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:49.176772 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-58779d2f-8a6e-4e58-90b8-5d4e59977dd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a25fd7b402e41068a7353de62e76d8c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:49.177683 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-58779d2f-8a6e-4e58-90b8-5d4e59977dd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a25fd7b402e41068a7353de62e76d8c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:49.186491 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-58779d2f-8a6e-4e58-90b8-5d4e59977dd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a25fd7b402e41068a7353de62e76d8c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:49.204243 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 886/1796] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:25:48 2022] DELETE /identity/v3/users/0a25fd7b402e41068a7353de62e76d8c => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:49.208700 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82504f26-620e-43e4-b46c-0a367a07bae7 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:49.209134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82504f26-620e-43e4-b46c-0a367a07bae7 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:49.209504 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82504f26-620e-43e4-b46c-0a367a07bae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:49.242300 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-82504f26-620e-43e4-b46c-0a367a07bae7 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:49.243747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-82504f26-620e-43e4-b46c-0a367a07bae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:49.318253 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 911/1797] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:49.343479 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58779d2f-8a6e-4e58-90b8-5d4e59977dd3 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:49.345984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-58779d2f-8a6e-4e58-90b8-5d4e59977dd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:49.439078 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58779d2f-8a6e-4e58-90b8-5d4e59977dd3 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:49.444539 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7ed02-342a-4593-9df3-0a5ea35a0654 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:49.445297 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7ed02-342a-4593-9df3-0a5ea35a0654 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:49.446279 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7ed02-342a-4593-9df3-0a5ea35a0654 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:49.446631 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7ed02-342a-4593-9df3-0a5ea35a0654 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:49.455344 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:49.459719 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e7ed02-342a-4593-9df3-0a5ea35a0654 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:49.461599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e7ed02-342a-4593-9df3-0a5ea35a0654 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:49.477540 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 887/1798] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:50.297867 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82504f26-620e-43e4-b46c-0a367a07bae7 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:50.299959 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-82504f26-620e-43e4-b46c-0a367a07bae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:50.314320 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82504f26-620e-43e4-b46c-0a367a07bae7 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:50.319218 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd189e-db00-426b-8bd0-2d9dff9d99d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:50.319871 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd189e-db00-426b-8bd0-2d9dff9d99d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:50.320150 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd189e-db00-426b-8bd0-2d9dff9d99d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:50.320440 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd189e-db00-426b-8bd0-2d9dff9d99d3 admin admin] PATH_INFO: `/v3/projects/0cde47c8423348408353e74a792d98b1` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:50.321599 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd189e-db00-426b-8bd0-2d9dff9d99d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cde47c8423348408353e74a792d98b1)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:50.323722 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd189e-db00-426b-8bd0-2d9dff9d99d3 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:50.888226 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-e3fd189e-db00-426b-8bd0-2d9dff9d99d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cde47c8423348408353e74a792d98b1', 'request_id': 'req-e3fd189e-db00-426b-8bd0-2d9dff9d99d3'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:50.896747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-e3fd189e-db00-426b-8bd0-2d9dff9d99d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cde47c8423348408353e74a792d98b1', 'request_id': 'req-e3fd189e-db00-426b-8bd0-2d9dff9d99d3'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:50.904209 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-e3fd189e-db00-426b-8bd0-2d9dff9d99d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cde47c8423348408353e74a792d98b1', 'request_id': 'req-e3fd189e-db00-426b-8bd0-2d9dff9d99d3'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:51.045283 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-e3fd189e-db00-426b-8bd0-2d9dff9d99d3 admin admin] The token cache is being invalidate because project 0cde47c8423348408353e74a792d98b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:25:51.045632 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-e3fd189e-db00-426b-8bd0-2d9dff9d99d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:51.046312 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-e3fd189e-db00-426b-8bd0-2d9dff9d99d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:51.052261 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7ed02-342a-4593-9df3-0a5ea35a0654 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:51.052812 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 912/1799] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:25:50 2022] DELETE /identity/v3/projects/0cde47c8423348408353e74a792d98b1 => generated 0 bytes in 751 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:51.068586 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7ed02-342a-4593-9df3-0a5ea35a0654 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:51.133230 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a8a6f-8324-49c1-8d7c-8d1ddd9060d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:51.134624 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a8a6f-8324-49c1-8d7c-8d1ddd9060d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:51.135080 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a8a6f-8324-49c1-8d7c-8d1ddd9060d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:51.135365 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a8a6f-8324-49c1-8d7c-8d1ddd9060d1 admin admin] PATH_INFO: `/v3/users/9d041dfc595f42caacb4066221c07d95` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:51.137500 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a8a6f-8324-49c1-8d7c-8d1ddd9060d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d041dfc595f42caacb4066221c07d95)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:51.141864 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a8a6f-8324-49c1-8d7c-8d1ddd9060d1 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:51.311767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd189e-db00-426b-8bd0-2d9dff9d99d3 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:51.314208 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-e3fd189e-db00-426b-8bd0-2d9dff9d99d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:51.338783 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd189e-db00-426b-8bd0-2d9dff9d99d3 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:51.349114 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60daf244-9fba-4466-8909-6bfd4571f810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:51.349784 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60daf244-9fba-4466-8909-6bfd4571f810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:51.350121 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60daf244-9fba-4466-8909-6bfd4571f810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:51.350842 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60daf244-9fba-4466-8909-6bfd4571f810 admin admin] PATH_INFO: `/v3/projects/a5015f41d4124d9998d47edb24f25660` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:51.352354 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60daf244-9fba-4466-8909-6bfd4571f810 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5015f41d4124d9998d47edb24f25660)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:51.352544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c30a8a6f-8324-49c1-8d7c-8d1ddd9060d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d041dfc595f42caacb4066221c07d95'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:51.360216 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60daf244-9fba-4466-8909-6bfd4571f810 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:51.376000 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c30a8a6f-8324-49c1-8d7c-8d1ddd9060d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d041dfc595f42caacb4066221c07d95'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:51.429403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c30a8a6f-8324-49c1-8d7c-8d1ddd9060d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d041dfc595f42caacb4066221c07d95'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:51.430124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c30a8a6f-8324-49c1-8d7c-8d1ddd9060d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d041dfc595f42caacb4066221c07d95'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:51.437632 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c30a8a6f-8324-49c1-8d7c-8d1ddd9060d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d041dfc595f42caacb4066221c07d95'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:51.443558 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 888/1800] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:25:51 2022] DELETE /identity/v3/users/9d041dfc595f42caacb4066221c07d95 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:51.506319 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-60daf244-9fba-4466-8909-6bfd4571f810 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5015f41d4124d9998d47edb24f25660', 'request_id': 'req-60daf244-9fba-4466-8909-6bfd4571f810'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:51.513142 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-60daf244-9fba-4466-8909-6bfd4571f810 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5015f41d4124d9998d47edb24f25660', 'request_id': 'req-60daf244-9fba-4466-8909-6bfd4571f810'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:51.519832 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:51.524912 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-60daf244-9fba-4466-8909-6bfd4571f810 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5015f41d4124d9998d47edb24f25660', 'request_id': 'req-60daf244-9fba-4466-8909-6bfd4571f810'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:51.631044 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-60daf244-9fba-4466-8909-6bfd4571f810 admin admin] The token cache is being invalidate because project a5015f41d4124d9998d47edb24f25660 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:25:51.631397 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-60daf244-9fba-4466-8909-6bfd4571f810 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:51.632140 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-60daf244-9fba-4466-8909-6bfd4571f810 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:51.634342 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 913/1801] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:25:51 2022] DELETE /identity/v3/projects/a5015f41d4124d9998d47edb24f25660 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:51.638061 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a8a6f-8324-49c1-8d7c-8d1ddd9060d1 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:51.639623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c30a8a6f-8324-49c1-8d7c-8d1ddd9060d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:51.705850 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a8a6f-8324-49c1-8d7c-8d1ddd9060d1 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:51.783739 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7114f-29af-4552-8383-c5f4b47330c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:51.785636 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7114f-29af-4552-8383-c5f4b47330c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:51.786859 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7114f-29af-4552-8383-c5f4b47330c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:51.787168 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7114f-29af-4552-8383-c5f4b47330c2 admin admin] PATH_INFO: `/v3/users/3b14c825c9d84a7993e6ba2a6f1c0bd2` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:51.788631 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc7114f-29af-4552-8383-c5f4b47330c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b14c825c9d84a7993e6ba2a6f1c0bd2)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:51.790774 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc7114f-29af-4552-8383-c5f4b47330c2 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:51.929126 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60daf244-9fba-4466-8909-6bfd4571f810 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:51.930631 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-60daf244-9fba-4466-8909-6bfd4571f810 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:51.953620 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60daf244-9fba-4466-8909-6bfd4571f810 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:51.956656 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb5491-6772-4b93-9436-81d2993dfa27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:51.957464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb5491-6772-4b93-9436-81d2993dfa27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:51.957816 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb5491-6772-4b93-9436-81d2993dfa27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:51.958160 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb5491-6772-4b93-9436-81d2993dfa27 admin admin] PATH_INFO: `/v3/projects/3fa111a0a0fc48459268a62f8b48915f` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:51.959529 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fb5491-6772-4b93-9436-81d2993dfa27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fa111a0a0fc48459268a62f8b48915f)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:51.959964 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-0dc7114f-29af-4552-8383-c5f4b47330c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b14c825c9d84a7993e6ba2a6f1c0bd2'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:51.961747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fb5491-6772-4b93-9436-81d2993dfa27 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:51.974202 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-0dc7114f-29af-4552-8383-c5f4b47330c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b14c825c9d84a7993e6ba2a6f1c0bd2'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:52.058563 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-0dc7114f-29af-4552-8383-c5f4b47330c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b14c825c9d84a7993e6ba2a6f1c0bd2'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:52.059817 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-0dc7114f-29af-4552-8383-c5f4b47330c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b14c825c9d84a7993e6ba2a6f1c0bd2'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:52.068340 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-0dc7114f-29af-4552-8383-c5f4b47330c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b14c825c9d84a7993e6ba2a6f1c0bd2'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:52.074929 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 889/1802] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:25:51 2022] DELETE /identity/v3/users/3b14c825c9d84a7993e6ba2a6f1c0bd2 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:52.144032 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-e3fb5491-6772-4b93-9436-81d2993dfa27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fa111a0a0fc48459268a62f8b48915f', 'request_id': 'req-e3fb5491-6772-4b93-9436-81d2993dfa27'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:52.154012 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-e3fb5491-6772-4b93-9436-81d2993dfa27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fa111a0a0fc48459268a62f8b48915f', 'request_id': 'req-e3fb5491-6772-4b93-9436-81d2993dfa27'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:52.161040 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-e3fb5491-6772-4b93-9436-81d2993dfa27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fa111a0a0fc48459268a62f8b48915f', 'request_id': 'req-e3fb5491-6772-4b93-9436-81d2993dfa27'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:52.276778 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-e3fb5491-6772-4b93-9436-81d2993dfa27 admin admin] The token cache is being invalidate because project 3fa111a0a0fc48459268a62f8b48915f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:25:52.277838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-e3fb5491-6772-4b93-9436-81d2993dfa27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:52.279757 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-e3fb5491-6772-4b93-9436-81d2993dfa27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:52.282918 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 914/1803] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:25:51 2022] DELETE /identity/v3/projects/3fa111a0a0fc48459268a62f8b48915f => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:52.287667 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7db8c-2684-4fdd-8fca-b2eb795651e4 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:52.287959 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7db8c-2684-4fdd-8fca-b2eb795651e4 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:52.288239 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7db8c-2684-4fdd-8fca-b2eb795651e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:52.322838 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-91c7db8c-2684-4fdd-8fca-b2eb795651e4 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:52.324141 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-91c7db8c-2684-4fdd-8fca-b2eb795651e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:52.405574 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 890/1804] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:52.412679 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dce4d4-2ccd-404d-b0ca-85da06bfeec3 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:52.413201 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dce4d4-2ccd-404d-b0ca-85da06bfeec3 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:52.413201 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dce4d4-2ccd-404d-b0ca-85da06bfeec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:52.436237 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-d5dce4d4-2ccd-404d-b0ca-85da06bfeec3 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:52.437746 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-d5dce4d4-2ccd-404d-b0ca-85da06bfeec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:52.446933 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 915/1805] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:52.451422 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7db8c-2684-4fdd-8fca-b2eb795651e4 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:52.463814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7db8c-2684-4fdd-8fca-b2eb795651e4 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:52.469267 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de980b-2ca4-4f4c-a032-139c0b504ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:52.470052 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de980b-2ca4-4f4c-a032-139c0b504ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:52.470383 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de980b-2ca4-4f4c-a032-139c0b504ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:52.470719 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de980b-2ca4-4f4c-a032-139c0b504ee3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:52.471483 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7de980b-2ca4-4f4c-a032-139c0b504ee3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:25:52.471916 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7de980b-2ca4-4f4c-a032-139c0b504ee3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:52.474838 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7de980b-2ca4-4f4c-a032-139c0b504ee3 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:52.487473 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 891/1806] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:25:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:25:52.498228 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dce4d4-2ccd-404d-b0ca-85da06bfeec3 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:52.515729 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dce4d4-2ccd-404d-b0ca-85da06bfeec3 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:52.521469 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa4bff-2ba8-43a2-a01e-1def2263cfd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:52.522367 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa4bff-2ba8-43a2-a01e-1def2263cfd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:52.522851 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa4bff-2ba8-43a2-a01e-1def2263cfd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:52.523326 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa4bff-2ba8-43a2-a01e-1def2263cfd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:52.524194 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fa4bff-2ba8-43a2-a01e-1def2263cfd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:52.526608 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fa4bff-2ba8-43a2-a01e-1def2263cfd1 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:52.555433 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:52.619778 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de980b-2ca4-4f4c-a032-139c0b504ee3 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:52.621585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d7de980b-2ca4-4f4c-a032-139c0b504ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:52.639008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 916/1807] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:25:52 2022] POST /identity/v3/projects => generated 401 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:52.640564 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa4bff-2ba8-43a2-a01e-1def2263cfd1 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:52.643731 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de980b-2ca4-4f4c-a032-139c0b504ee3 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:52.721917 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa4bff-2ba8-43a2-a01e-1def2263cfd1 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:52.727654 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831bb337-3a00-4405-a1eb-b38dfd5f8ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:52.728274 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bb337-3a00-4405-a1eb-b38dfd5f8ff8 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:52.728727 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bb337-3a00-4405-a1eb-b38dfd5f8ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:52.729006 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bb337-3a00-4405-a1eb-b38dfd5f8ff8 admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:52.729737 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831bb337-3a00-4405-a1eb-b38dfd5f8ff8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:52.730956 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6983071b-bb12-4107-92ad-3df0a961c7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:52.731603 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6983071b-bb12-4107-92ad-3df0a961c7af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:52.731784 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831bb337-3a00-4405-a1eb-b38dfd5f8ff8 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:52.731879 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6983071b-bb12-4107-92ad-3df0a961c7af admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:52.732130 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6983071b-bb12-4107-92ad-3df0a961c7af admin admin] PATH_INFO: `/v3/projects/86b92b44328c4c41aae7fb4728305ad8` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:52.733621 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6983071b-bb12-4107-92ad-3df0a961c7af admin admin] RBAC: Authorizing `identity:delete_project(project_id=86b92b44328c4c41aae7fb4728305ad8)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:52.735644 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6983071b-bb12-4107-92ad-3df0a961c7af admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:52.839899 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 917/1808] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:25:52 2022] POST /identity/v3/users => generated 301 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:52.844131 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831bb337-3a00-4405-a1eb-b38dfd5f8ff8 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:52.853751 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831bb337-3a00-4405-a1eb-b38dfd5f8ff8 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:52.858545 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305aa4d5-6364-42d7-9d5c-fcf7db9f8d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:52.859129 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305aa4d5-6364-42d7-9d5c-fcf7db9f8d03 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:52.859407 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305aa4d5-6364-42d7-9d5c-fcf7db9f8d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:52.859677 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305aa4d5-6364-42d7-9d5c-fcf7db9f8d03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:52.860434 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305aa4d5-6364-42d7-9d5c-fcf7db9f8d03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:52.862173 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305aa4d5-6364-42d7-9d5c-fcf7db9f8d03 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:52.873439 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 918/1809] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:25:52.877372 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305aa4d5-6364-42d7-9d5c-fcf7db9f8d03 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:52.888522 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305aa4d5-6364-42d7-9d5c-fcf7db9f8d03 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:52.890983 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-6983071b-bb12-4107-92ad-3df0a961c7af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86b92b44328c4c41aae7fb4728305ad8', 'request_id': 'req-6983071b-bb12-4107-92ad-3df0a961c7af'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:52.901862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-6983071b-bb12-4107-92ad-3df0a961c7af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86b92b44328c4c41aae7fb4728305ad8', 'request_id': 'req-6983071b-bb12-4107-92ad-3df0a961c7af'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:52.955674 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa821ccd-55f8-419e-bdeb-882ccd7ea710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:52.956302 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa821ccd-55f8-419e-bdeb-882ccd7ea710 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:52.957221 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa821ccd-55f8-419e-bdeb-882ccd7ea710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:52.957696 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa821ccd-55f8-419e-bdeb-882ccd7ea710 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:52.958592 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa821ccd-55f8-419e-bdeb-882ccd7ea710 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:52.960216 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa821ccd-55f8-419e-bdeb-882ccd7ea710 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:52.972073 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 919/1810] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:52 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:25:52.972446 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-6983071b-bb12-4107-92ad-3df0a961c7af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86b92b44328c4c41aae7fb4728305ad8', 'request_id': 'req-6983071b-bb12-4107-92ad-3df0a961c7af'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:52.981930 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa821ccd-55f8-419e-bdeb-882ccd7ea710 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:52.987689 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-6983071b-bb12-4107-92ad-3df0a961c7af admin admin] The token cache is being invalidate because project 86b92b44328c4c41aae7fb4728305ad8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:25:52.988079 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-6983071b-bb12-4107-92ad-3df0a961c7af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:52.989348 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-6983071b-bb12-4107-92ad-3df0a961c7af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:52.991234 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 892/1811] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:25:52 2022] DELETE /identity/v3/projects/86b92b44328c4c41aae7fb4728305ad8 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:25:52.995858 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb6ad7-ed5b-4f76-a634-a6d6c7b203c5 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:52.996141 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb6ad7-ed5b-4f76-a634-a6d6c7b203c5 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:52.996429 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb6ad7-ed5b-4f76-a634-a6d6c7b203c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:53.008318 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa821ccd-55f8-419e-bdeb-882ccd7ea710 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:53.023931 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d13a02-8b23-4805-9d2a-eea576896001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:53.024638 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d13a02-8b23-4805-9d2a-eea576896001 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:53.025553 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d13a02-8b23-4805-9d2a-eea576896001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:53.025845 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d13a02-8b23-4805-9d2a-eea576896001 admin admin] PATH_INFO: `/v3/projects/d968144847784badba4ad35e55c20777/users/24f59721b45943378ca6fcc9d3a5723c/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:53.035378 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-ebcb6ad7-ed5b-4f76-a634-a6d6c7b203c5 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:53.035378 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ebcb6ad7-ed5b-4f76-a634-a6d6c7b203c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:53.049706 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d13a02-8b23-4805-9d2a-eea576896001 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d968144847784badba4ad35e55c20777, user_id=24f59721b45943378ca6fcc9d3a5723c, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:53.050293 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 893/1812] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:25:53.058821 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bcd84b-7512-4faa-8649-0d972d4c06c2 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:53.059376 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bcd84b-7512-4faa-8649-0d972d4c06c2 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:53.059376 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bcd84b-7512-4faa-8649-0d972d4c06c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:53.065307 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d13a02-8b23-4805-9d2a-eea576896001 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:53.083808 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-59bcd84b-7512-4faa-8649-0d972d4c06c2 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:53.085237 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-59bcd84b-7512-4faa-8649-0d972d4c06c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:53.093849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 894/1813] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:53.099271 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bcd84b-7512-4faa-8649-0d972d4c06c2 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:53.108195 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bcd84b-7512-4faa-8649-0d972d4c06c2 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:53.127000 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310629fe-e20c-4092-9dbd-6dd019d335e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:53.127751 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310629fe-e20c-4092-9dbd-6dd019d335e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:53.128124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310629fe-e20c-4092-9dbd-6dd019d335e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:53.129155 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310629fe-e20c-4092-9dbd-6dd019d335e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:53.131155 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310629fe-e20c-4092-9dbd-6dd019d335e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:25:53.131457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310629fe-e20c-4092-9dbd-6dd019d335e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:53.133981 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310629fe-e20c-4092-9dbd-6dd019d335e5 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:53.153115 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 920/1814] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:25:52 2022] PUT /identity/v3/projects/d968144847784badba4ad35e55c20777/users/24f59721b45943378ca6fcc9d3a5723c/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:25:53.156603 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 895/1815] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:25:53.160576 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d13a02-8b23-4805-9d2a-eea576896001 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:53.167059 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7cf14-bdfb-44a3-9e72-cb198007e805 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:53.167348 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7cf14-bdfb-44a3-9e72-cb198007e805 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:53.168833 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7cf14-bdfb-44a3-9e72-cb198007e805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:53.210383 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-85e7cf14-bdfb-44a3-9e72-cb198007e805 None None] MFA Rules not processed for user `24f59721b45943378ca6fcc9d3a5723c`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:53.211954 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-85e7cf14-bdfb-44a3-9e72-cb198007e805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:53.272460 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d13a02-8b23-4805-9d2a-eea576896001 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:53.278153 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace6634-5657-4928-9338-38dd0a7dce5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:53.281283 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace6634-5657-4928-9338-38dd0a7dce5c admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:53.281581 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace6634-5657-4928-9338-38dd0a7dce5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:53.281972 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace6634-5657-4928-9338-38dd0a7dce5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:53.282566 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace6634-5657-4928-9338-38dd0a7dce5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:53.285612 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace6634-5657-4928-9338-38dd0a7dce5c admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:53.310934 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 896/1816] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:53 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:53.315396 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7cf14-bdfb-44a3-9e72-cb198007e805 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:53.317085 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-85e7cf14-bdfb-44a3-9e72-cb198007e805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:53.333746 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7cf14-bdfb-44a3-9e72-cb198007e805 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:53.338908 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6995cdc1-2750-4023-ac2b-cc15f40c847d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:53.339557 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6995cdc1-2750-4023-ac2b-cc15f40c847d admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:53.339813 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6995cdc1-2750-4023-ac2b-cc15f40c847d admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:53.340089 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6995cdc1-2750-4023-ac2b-cc15f40c847d admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:53.340819 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6995cdc1-2750-4023-ac2b-cc15f40c847d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:53.343028 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6995cdc1-2750-4023-ac2b-cc15f40c847d admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:53.408552 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 921/1817] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:25:53 2022] POST /identity/v3/projects => generated 387 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:53.413332 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace6634-5657-4928-9338-38dd0a7dce5c admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:53.468465 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 897/1818] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:25:53 2022] POST /identity/v3/projects => generated 397 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:53.481021 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6995cdc1-2750-4023-ac2b-cc15f40c847d admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:53.527428 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace6634-5657-4928-9338-38dd0a7dce5c admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:53.674870 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6995cdc1-2750-4023-ac2b-cc15f40c847d admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:53.676860 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:DELETE=9,keystone:SELECT=85,keystone:INSERT=6 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:53.685180 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebba068-6692-4a99-b11b-a316b5d9a860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:53.685823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebba068-6692-4a99-b11b-a316b5d9a860 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:53.686100 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebba068-6692-4a99-b11b-a316b5d9a860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:53.686376 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebba068-6692-4a99-b11b-a316b5d9a860 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:53.687046 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebba068-6692-4a99-b11b-a316b5d9a860 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:53.692529 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6705cff2-457e-42c0-be83-f648f5045e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:53.693179 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebba068-6692-4a99-b11b-a316b5d9a860 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:53.693613 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6705cff2-457e-42c0-be83-f648f5045e82 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:53.694490 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6705cff2-457e-42c0-be83-f648f5045e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:53.694905 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6705cff2-457e-42c0-be83-f648f5045e82 admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:53.695686 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6705cff2-457e-42c0-be83-f648f5045e82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:53.698590 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6705cff2-457e-42c0-be83-f648f5045e82 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:53.848574 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 898/1819] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:25:53 2022] POST /identity/v3/users => generated 305 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:25:53.852720 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebba068-6692-4a99-b11b-a316b5d9a860 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:53.868251 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebba068-6692-4a99-b11b-a316b5d9a860 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:53.873429 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f159336-290b-40ef-84fc-088d155ac12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:53.880364 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f159336-290b-40ef-84fc-088d155ac12e admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:53.880364 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f159336-290b-40ef-84fc-088d155ac12e admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:53.880364 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f159336-290b-40ef-84fc-088d155ac12e admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:53.880364 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f159336-290b-40ef-84fc-088d155ac12e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:53.880364 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f159336-290b-40ef-84fc-088d155ac12e admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:53.880774 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 922/1820] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:25:53 2022] POST /identity/v3/users => generated 294 bytes in 467 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:25:53.883058 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6705cff2-457e-42c0-be83-f648f5045e82 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:53.903132 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 899/1821] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:25:53.906041 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f159336-290b-40ef-84fc-088d155ac12e admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:53.920341 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6705cff2-457e-42c0-be83-f648f5045e82 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:53.921386 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f159336-290b-40ef-84fc-088d155ac12e admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:53.929862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2117a-05fd-4a5e-89c8-d72cb1c87afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:53.931339 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2117a-05fd-4a5e-89c8-d72cb1c87afa admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:53.931501 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b32a6-83b3-4a56-84af-eb38bcae632f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:53.931826 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2117a-05fd-4a5e-89c8-d72cb1c87afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:53.931986 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b32a6-83b3-4a56-84af-eb38bcae632f admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:53.932272 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b32a6-83b3-4a56-84af-eb38bcae632f admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:53.932524 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2117a-05fd-4a5e-89c8-d72cb1c87afa admin admin] PATH_INFO: `/v3/projects/53c0ecad63494c169c6cce1fcbdac30c/users/429b5bf1eccd442a8aa5f561440de79a/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:53.932632 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b32a6-83b3-4a56-84af-eb38bcae632f admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:53.933728 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727b32a6-83b3-4a56-84af-eb38bcae632f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:53.934981 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727b32a6-83b3-4a56-84af-eb38bcae632f admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:53.967822 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 923/1822] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:25:53.970084 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b32a6-83b3-4a56-84af-eb38bcae632f admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:53.979991 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a2117a-05fd-4a5e-89c8-d72cb1c87afa admin admin] RBAC: Authorizing `identity:create_grant(project_id=53c0ecad63494c169c6cce1fcbdac30c, user_id=429b5bf1eccd442a8aa5f561440de79a, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:53.983037 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a2117a-05fd-4a5e-89c8-d72cb1c87afa admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:53.983182 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b32a6-83b3-4a56-84af-eb38bcae632f admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:53.989676 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c051839-44a0-4c8f-9f57-987caee67d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:53.991327 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c051839-44a0-4c8f-9f57-987caee67d2f admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:53.991617 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c051839-44a0-4c8f-9f57-987caee67d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:53.991899 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c051839-44a0-4c8f-9f57-987caee67d2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:53.992641 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c051839-44a0-4c8f-9f57-987caee67d2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:53.994531 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c051839-44a0-4c8f-9f57-987caee67d2f admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:54.010837 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 924/1823] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:25:54.014971 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c051839-44a0-4c8f-9f57-987caee67d2f admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:54.025272 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c051839-44a0-4c8f-9f57-987caee67d2f admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:54.030847 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604de9e0-6be1-4455-9bf4-90df62e76f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:54.031623 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604de9e0-6be1-4455-9bf4-90df62e76f46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:54.032178 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604de9e0-6be1-4455-9bf4-90df62e76f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:54.032276 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604de9e0-6be1-4455-9bf4-90df62e76f46 admin admin] PATH_INFO: `/v3/projects/0aad2d43c2ac4da18ecf568e499fd6eb/users/5caa43a66b43462fa043685913e7fa58/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:54.047896 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604de9e0-6be1-4455-9bf4-90df62e76f46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0aad2d43c2ac4da18ecf568e499fd6eb, user_id=5caa43a66b43462fa043685913e7fa58, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:54.051368 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604de9e0-6be1-4455-9bf4-90df62e76f46 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:54.086846 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 900/1824] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:25:53 2022] PUT /identity/v3/projects/53c0ecad63494c169c6cce1fcbdac30c/users/429b5bf1eccd442a8aa5f561440de79a/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:54.090799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2117a-05fd-4a5e-89c8-d72cb1c87afa admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:54.156273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2117a-05fd-4a5e-89c8-d72cb1c87afa admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:54.161168 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85560b29-4dd7-4b21-b6e7-c73646b12efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:54.161757 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85560b29-4dd7-4b21-b6e7-c73646b12efa admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:54.162022 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85560b29-4dd7-4b21-b6e7-c73646b12efa admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:54.162443 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85560b29-4dd7-4b21-b6e7-c73646b12efa admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:54.163008 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85560b29-4dd7-4b21-b6e7-c73646b12efa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:54.164786 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85560b29-4dd7-4b21-b6e7-c73646b12efa admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:54.175165 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 901/1825] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:25:54.184975 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85560b29-4dd7-4b21-b6e7-c73646b12efa admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:54.197426 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85560b29-4dd7-4b21-b6e7-c73646b12efa admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:54.203641 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287e30f5-fa6c-4f35-963f-0c0a723125e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:54.204404 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e30f5-fa6c-4f35-963f-0c0a723125e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:54.205995 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e30f5-fa6c-4f35-963f-0c0a723125e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:54.205995 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e30f5-fa6c-4f35-963f-0c0a723125e5 admin admin] PATH_INFO: `/v3/projects/53c0ecad63494c169c6cce1fcbdac30c/users/429b5bf1eccd442a8aa5f561440de79a/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:54.208243 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287e30f5-fa6c-4f35-963f-0c0a723125e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53c0ecad63494c169c6cce1fcbdac30c, user_id=429b5bf1eccd442a8aa5f561440de79a, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:54.211104 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287e30f5-fa6c-4f35-963f-0c0a723125e5 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:54.223074 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 902/1826] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:25:54 2022] PUT /identity/v3/projects/53c0ecad63494c169c6cce1fcbdac30c/users/429b5bf1eccd442a8aa5f561440de79a/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:54.234444 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76127a2c-5824-4173-817e-b0e86b922feb None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:54.234749 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76127a2c-5824-4173-817e-b0e86b922feb None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:54.234997 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76127a2c-5824-4173-817e-b0e86b922feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:54.236783 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 925/1827] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:25:54 2022] PUT /identity/v3/projects/0aad2d43c2ac4da18ecf568e499fd6eb/users/5caa43a66b43462fa043685913e7fa58/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:25:54.250808 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4558c-5676-496a-9638-b118d9c2d4a4 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:54.251099 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4558c-5676-496a-9638-b118d9c2d4a4 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:54.251389 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4558c-5676-496a-9638-b118d9c2d4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:54.291280 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-76127a2c-5824-4173-817e-b0e86b922feb None None] MFA Rules not processed for user `429b5bf1eccd442a8aa5f561440de79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:54.292679 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-76127a2c-5824-4173-817e-b0e86b922feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:54.340853 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-d8c4558c-5676-496a-9638-b118d9c2d4a4 None None] MFA Rules not processed for user `5caa43a66b43462fa043685913e7fa58`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:54.344619 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-d8c4558c-5676-496a-9638-b118d9c2d4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:54.423375 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 903/1828] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:54 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:25:54.436239 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76127a2c-5824-4173-817e-b0e86b922feb None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:54.437969 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-76127a2c-5824-4173-817e-b0e86b922feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:54.475309 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 926/1829] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:54 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:25:54.489224 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4558c-5676-496a-9638-b118d9c2d4a4 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:54.519251 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76127a2c-5824-4173-817e-b0e86b922feb None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:54.525203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc816d0-bcf9-4814-b1ed-bba77400ccf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:54.525843 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc816d0-bcf9-4814-b1ed-bba77400ccf9 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:54.526124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc816d0-bcf9-4814-b1ed-bba77400ccf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:54.526391 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc816d0-bcf9-4814-b1ed-bba77400ccf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:54.568385 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc816d0-bcf9-4814-b1ed-bba77400ccf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:54.569878 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc816d0-bcf9-4814-b1ed-bba77400ccf9 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:54.598588 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 904/1830] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:54 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 14:25:54.606912 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:54.648983 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4558c-5676-496a-9638-b118d9c2d4a4 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:54.654331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba7d21-21c2-424a-89aa-13dfdb802683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:54.655043 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba7d21-21c2-424a-89aa-13dfdb802683 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:54.655375 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba7d21-21c2-424a-89aa-13dfdb802683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:54.655838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba7d21-21c2-424a-89aa-13dfdb802683 admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:54.656553 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ba7d21-21c2-424a-89aa-13dfdb802683 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:54.658701 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ba7d21-21c2-424a-89aa-13dfdb802683 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:54.751178 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc816d0-bcf9-4814-b1ed-bba77400ccf9 service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:54.752988 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ccc816d0-bcf9-4814-b1ed-bba77400ccf9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:54.766868 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc816d0-bcf9-4814-b1ed-bba77400ccf9 service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:54.775711 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb73b650-5e71-4c6d-8d8d-6ec791149fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:54.775711 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73b650-5e71-4c6d-8d8d-6ec791149fac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:54.775711 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73b650-5e71-4c6d-8d8d-6ec791149fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:54.775711 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73b650-5e71-4c6d-8d8d-6ec791149fac admin admin] PATH_INFO: `/v3/users/9475bab03724429b8273fc9e3c1bc7df` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:54.776032 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb73b650-5e71-4c6d-8d8d-6ec791149fac admin admin] RBAC: Authorizing `identity:delete_user(user_id=9475bab03724429b8273fc9e3c1bc7df)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:54.776883 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb73b650-5e71-4c6d-8d8d-6ec791149fac admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:54.821999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 927/1831] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:25:54 2022] POST /identity/v3/projects => generated 387 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:54.825848 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba7d21-21c2-424a-89aa-13dfdb802683 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:54.891926 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba7d21-21c2-424a-89aa-13dfdb802683 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:54.897592 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b7a60-28c6-453b-bcc8-ef552352a382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:54.898211 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b7a60-28c6-453b-bcc8-ef552352a382 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:54.898490 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b7a60-28c6-453b-bcc8-ef552352a382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:54.898762 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b7a60-28c6-453b-bcc8-ef552352a382 admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:54.899439 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926b7a60-28c6-453b-bcc8-ef552352a382 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:54.901390 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926b7a60-28c6-453b-bcc8-ef552352a382 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:55.021504 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 928/1832] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:25:54 2022] POST /identity/v3/users => generated 300 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:25:55.022339 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b7a60-28c6-453b-bcc8-ef552352a382 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:55.031867 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-bb73b650-5e71-4c6d-8d8d-6ec791149fac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9475bab03724429b8273fc9e3c1bc7df'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:55.062919 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-bb73b650-5e71-4c6d-8d8d-6ec791149fac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9475bab03724429b8273fc9e3c1bc7df'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:55.107617 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b7a60-28c6-453b-bcc8-ef552352a382 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:55.111328 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d791f801-fa68-4a94-8a1f-1c014091036f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:55.112070 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791f801-fa68-4a94-8a1f-1c014091036f service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:55.112356 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791f801-fa68-4a94-8a1f-1c014091036f service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:55.112722 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791f801-fa68-4a94-8a1f-1c014091036f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:55.144762 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-bb73b650-5e71-4c6d-8d8d-6ec791149fac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9475bab03724429b8273fc9e3c1bc7df'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:55.146062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-bb73b650-5e71-4c6d-8d8d-6ec791149fac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9475bab03724429b8273fc9e3c1bc7df'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:55.154628 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-bb73b650-5e71-4c6d-8d8d-6ec791149fac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9475bab03724429b8273fc9e3c1bc7df'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:55.164235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 905/1833] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:25:54 2022] DELETE /identity/v3/users/9475bab03724429b8273fc9e3c1bc7df => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:25:55.165293 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb73b650-5e71-4c6d-8d8d-6ec791149fac admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:55.182778 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-bb73b650-5e71-4c6d-8d8d-6ec791149fac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:55.189903 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d791f801-fa68-4a94-8a1f-1c014091036f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:55.193842 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d791f801-fa68-4a94-8a1f-1c014091036f service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:55.204301 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 929/1834] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:55 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:55.205439 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b2438-0de5-490b-b406-1188579faae5 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:55.205889 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b2438-0de5-490b-b406-1188579faae5 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:55.206146 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b2438-0de5-490b-b406-1188579faae5 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:55.207896 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 930/1835] 10.222.0.26 () {54 vars in 921 bytes} [Fri Aug 5 14:25:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:25:55.209138 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89b866-7fab-4a12-8dd5-358181dad177 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:55.214251 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89b866-7fab-4a12-8dd5-358181dad177 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:55.214251 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89b866-7fab-4a12-8dd5-358181dad177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:55.246926 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-ea89b866-7fab-4a12-8dd5-358181dad177 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:55.248358 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-ea89b866-7fab-4a12-8dd5-358181dad177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:55.316598 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb73b650-5e71-4c6d-8d8d-6ec791149fac admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:55.322796 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc206f48-b5b0-49b1-bc4d-2a1860ce8938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:55.324249 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc206f48-b5b0-49b1-bc4d-2a1860ce8938 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:55.324249 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc206f48-b5b0-49b1-bc4d-2a1860ce8938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:55.324610 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc206f48-b5b0-49b1-bc4d-2a1860ce8938 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:55.325460 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc206f48-b5b0-49b1-bc4d-2a1860ce8938 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:55.327133 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc206f48-b5b0-49b1-bc4d-2a1860ce8938 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:55.329256 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 931/1836] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:55.335882 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc50de-e6eb-41a0-8a46-d164da91870c None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:55.336683 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc50de-e6eb-41a0-8a46-d164da91870c None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:55.338406 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc50de-e6eb-41a0-8a46-d164da91870c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:55.341498 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 906/1837] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:55.342850 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc206f48-b5b0-49b1-bc4d-2a1860ce8938 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:55.346728 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-dc206f48-b5b0-49b1-bc4d-2a1860ce8938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:55.370112 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-edbc50de-e6eb-41a0-8a46-d164da91870c None None] MFA Rules not processed for user `5ec4940c99fc4fa8a5bc7cdabc4558fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:55.371379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-edbc50de-e6eb-41a0-8a46-d164da91870c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:55.415772 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc206f48-b5b0-49b1-bc4d-2a1860ce8938 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:55.419530 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3d926-3c09-416a-bb8f-5fb839045386 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:55.421957 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3d926-3c09-416a-bb8f-5fb839045386 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:55.421957 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3d926-3c09-416a-bb8f-5fb839045386 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:55.421957 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3d926-3c09-416a-bb8f-5fb839045386 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:55.438545 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3d926-3c09-416a-bb8f-5fb839045386 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:55.440109 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3d926-3c09-416a-bb8f-5fb839045386 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:55.446560 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 932/1838] 10.222.0.26 () {58 vars in 1018 bytes} [Fri Aug 5 14:25:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:55.449343 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc50de-e6eb-41a0-8a46-d164da91870c None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:55.450572 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 907/1839] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:55.459142 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3d926-3c09-416a-bb8f-5fb839045386 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:55.460450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc50de-e6eb-41a0-8a46-d164da91870c None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:55.460627 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-64e3d926-3c09-416a-bb8f-5fb839045386 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:55.467471 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b679e1a0-a0c7-485e-9740-2e9c8e202dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:55.467471 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679e1a0-a0c7-485e-9740-2e9c8e202dc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:55.467471 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679e1a0-a0c7-485e-9740-2e9c8e202dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:55.467471 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679e1a0-a0c7-485e-9740-2e9c8e202dc9 admin admin] PATH_INFO: `/v3/projects/a3d9591db1f8459083c1c9ff5f316f63/users/68cad796a46f44eb9ed2c7a15cb59e8c/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:55.486524 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3d926-3c09-416a-bb8f-5fb839045386 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:55.487727 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b679e1a0-a0c7-485e-9740-2e9c8e202dc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3d9591db1f8459083c1c9ff5f316f63, user_id=68cad796a46f44eb9ed2c7a15cb59e8c, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:55.490749 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b679e1a0-a0c7-485e-9740-2e9c8e202dc9 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:55.493171 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac2afc3-2055-4a17-824b-9c9107d6ccf4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:55.493859 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac2afc3-2055-4a17-824b-9c9107d6ccf4 service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:55.494145 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac2afc3-2055-4a17-824b-9c9107d6ccf4 service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:55.496187 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac2afc3-2055-4a17-824b-9c9107d6ccf4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:55.506929 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac2afc3-2055-4a17-824b-9c9107d6ccf4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:55.508714 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac2afc3-2055-4a17-824b-9c9107d6ccf4 service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:55.522895 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 908/1840] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:55.602096 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 933/1841] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:25:55 2022] PUT /identity/v3/projects/a3d9591db1f8459083c1c9ff5f316f63/users/68cad796a46f44eb9ed2c7a15cb59e8c/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:55.606113 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac2afc3-2055-4a17-824b-9c9107d6ccf4 service nova] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:55.616992 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac2afc3-2055-4a17-824b-9c9107d6ccf4 service nova] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:55.675588 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54f05c-da4c-4faa-80df-3230b4a64734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:55.676230 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54f05c-da4c-4faa-80df-3230b4a64734 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:55.676536 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54f05c-da4c-4faa-80df-3230b4a64734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:55.676813 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54f05c-da4c-4faa-80df-3230b4a64734 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:55.677513 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54f05c-da4c-4faa-80df-3230b4a64734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:55.679120 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54f05c-da4c-4faa-80df-3230b4a64734 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:55.695354 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 909/1842] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:55.696525 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b679e1a0-a0c7-485e-9740-2e9c8e202dc9 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:55.711142 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b679e1a0-a0c7-485e-9740-2e9c8e202dc9 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:55.716677 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73fdba1-914a-49cb-b77f-f457e242e860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:55.717705 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fdba1-914a-49cb-b77f-f457e242e860 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:55.717975 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fdba1-914a-49cb-b77f-f457e242e860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:55.718253 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fdba1-914a-49cb-b77f-f457e242e860 admin admin] PATH_INFO: `/v3/projects/a3d9591db1f8459083c1c9ff5f316f63/users/68cad796a46f44eb9ed2c7a15cb59e8c/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:55.725272 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73fdba1-914a-49cb-b77f-f457e242e860 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3d9591db1f8459083c1c9ff5f316f63, user_id=68cad796a46f44eb9ed2c7a15cb59e8c, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:55.730021 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73fdba1-914a-49cb-b77f-f457e242e860 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:55.761506 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 934/1843] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:25:55 2022] PUT /identity/v3/projects/a3d9591db1f8459083c1c9ff5f316f63/users/68cad796a46f44eb9ed2c7a15cb59e8c/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:25:55.762653 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c1338-5ea4-46de-aff7-dc0e5d2b6441 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:55.762943 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c1338-5ea4-46de-aff7-dc0e5d2b6441 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:55.765094 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c1338-5ea4-46de-aff7-dc0e5d2b6441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:55.797121 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-290c1338-5ea4-46de-aff7-dc0e5d2b6441 None None] MFA Rules not processed for user `68cad796a46f44eb9ed2c7a15cb59e8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:55.799399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-290c1338-5ea4-46de-aff7-dc0e5d2b6441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:55.891600 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 910/1844] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:55 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:25:55.926820 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73fdba1-914a-49cb-b77f-f457e242e860 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:55.931671 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-f73fdba1-914a-49cb-b77f-f457e242e860 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:56.013775 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73fdba1-914a-49cb-b77f-f457e242e860 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:56.018196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5d5f8-94df-49a4-bfda-a60521b5a233 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:56.019484 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5d5f8-94df-49a4-bfda-a60521b5a233 service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:56.020028 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5d5f8-94df-49a4-bfda-a60521b5a233 service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:56.020393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5d5f8-94df-49a4-bfda-a60521b5a233 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:56.032521 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c5d5f8-94df-49a4-bfda-a60521b5a233 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:56.034104 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c5d5f8-94df-49a4-bfda-a60521b5a233 service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:56.044815 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 935/1845] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:55 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:56.141844 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:56.169879 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290c1338-5ea4-46de-aff7-dc0e5d2b6441 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:56.192041 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5d5f8-94df-49a4-bfda-a60521b5a233 service cinder] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:56.195376 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-d8c5d5f8-94df-49a4-bfda-a60521b5a233 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:56.274250 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290c1338-5ea4-46de-aff7-dc0e5d2b6441 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:56.287522 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d8314-22dc-44f9-bf5f-b63f74dac887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:56.288195 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d8314-22dc-44f9-bf5f-b63f74dac887 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:56.289101 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d8314-22dc-44f9-bf5f-b63f74dac887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:56.289101 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d8314-22dc-44f9-bf5f-b63f74dac887 admin admin] PATH_INFO: `/v3/users/5caa43a66b43462fa043685913e7fa58` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:56.291373 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3d8314-22dc-44f9-bf5f-b63f74dac887 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5caa43a66b43462fa043685913e7fa58)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:56.292120 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3d8314-22dc-44f9-bf5f-b63f74dac887 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:56.298595 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5d5f8-94df-49a4-bfda-a60521b5a233 service cinder] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:56.307014 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7228cad4-6cc3-4b10-956d-70d66efbd150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:56.307673 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228cad4-6cc3-4b10-956d-70d66efbd150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:56.307952 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228cad4-6cc3-4b10-956d-70d66efbd150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:56.308222 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228cad4-6cc3-4b10-956d-70d66efbd150 admin admin] PATH_INFO: `/v3/projects/587c3fc6a8f747509af8f09bfbabd673` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:56.309447 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7228cad4-6cc3-4b10-956d-70d66efbd150 admin admin] RBAC: Authorizing `identity:delete_project(project_id=587c3fc6a8f747509af8f09bfbabd673)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:56.311332 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7228cad4-6cc3-4b10-956d-70d66efbd150 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:56.564306 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-7228cad4-6cc3-4b10-956d-70d66efbd150 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '587c3fc6a8f747509af8f09bfbabd673', 'request_id': 'req-7228cad4-6cc3-4b10-956d-70d66efbd150'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:56.572920 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-7228cad4-6cc3-4b10-956d-70d66efbd150 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '587c3fc6a8f747509af8f09bfbabd673', 'request_id': 'req-7228cad4-6cc3-4b10-956d-70d66efbd150'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:56.579345 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-5f3d8314-22dc-44f9-bf5f-b63f74dac887 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5caa43a66b43462fa043685913e7fa58'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:56.598324 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-7228cad4-6cc3-4b10-956d-70d66efbd150 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '587c3fc6a8f747509af8f09bfbabd673', 'request_id': 'req-7228cad4-6cc3-4b10-956d-70d66efbd150'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:56.649287 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-5f3d8314-22dc-44f9-bf5f-b63f74dac887 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5caa43a66b43462fa043685913e7fa58'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:56.724303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-7228cad4-6cc3-4b10-956d-70d66efbd150 admin admin] The token cache is being invalidate because project 587c3fc6a8f747509af8f09bfbabd673 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:25:56.724655 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-7228cad4-6cc3-4b10-956d-70d66efbd150 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:56.726530 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-7228cad4-6cc3-4b10-956d-70d66efbd150 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:56.733985 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 936/1846] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:25:56 2022] DELETE /identity/v3/projects/587c3fc6a8f747509af8f09bfbabd673 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:56.749521 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee8fc2-b6c8-40d4-b817-43e6e38daf6a None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:56.749521 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee8fc2-b6c8-40d4-b817-43e6e38daf6a None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:56.749521 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee8fc2-b6c8-40d4-b817-43e6e38daf6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:56.781399 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-01ee8fc2-b6c8-40d4-b817-43e6e38daf6a None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:56.782760 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-01ee8fc2-b6c8-40d4-b817-43e6e38daf6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:56.801413 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-5f3d8314-22dc-44f9-bf5f-b63f74dac887 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5caa43a66b43462fa043685913e7fa58'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:56.803334 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-5f3d8314-22dc-44f9-bf5f-b63f74dac887 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5caa43a66b43462fa043685913e7fa58'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:56.815119 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-5f3d8314-22dc-44f9-bf5f-b63f74dac887 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5caa43a66b43462fa043685913e7fa58'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:56.828909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 911/1847] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:25:56 2022] DELETE /identity/v3/users/5caa43a66b43462fa043685913e7fa58 => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:56.830351 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a60b6-394f-4c8d-8d77-223795589831 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:56.830636 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a60b6-394f-4c8d-8d77-223795589831 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:56.833900 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a60b6-394f-4c8d-8d77-223795589831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:56.860556 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-665a60b6-394f-4c8d-8d77-223795589831 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:56.862091 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-665a60b6-394f-4c8d-8d77-223795589831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:56.929286 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 937/1848] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:56.941775 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96963a84-7a9f-4426-b9ca-f4f98f0282ff None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:56.942063 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96963a84-7a9f-4426-b9ca-f4f98f0282ff None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:56.944359 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96963a84-7a9f-4426-b9ca-f4f98f0282ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:56.992108 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-96963a84-7a9f-4426-b9ca-f4f98f0282ff None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:56.993636 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-96963a84-7a9f-4426-b9ca-f4f98f0282ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:57.025093 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 912/1849] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:57.050098 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a60b6-394f-4c8d-8d77-223795589831 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:57.053188 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-665a60b6-394f-4c8d-8d77-223795589831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:57.096509 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 938/1850] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:57.103926 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96963a84-7a9f-4426-b9ca-f4f98f0282ff None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:57.124571 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96963a84-7a9f-4426-b9ca-f4f98f0282ff None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:57.136614 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3615cfad-3621-434d-bb43-28b4c9f95aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:57.138103 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3615cfad-3621-434d-bb43-28b4c9f95aa0 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:57.143304 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3615cfad-3621-434d-bb43-28b4c9f95aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:57.148660 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3615cfad-3621-434d-bb43-28b4c9f95aa0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:57.151571 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3615cfad-3621-434d-bb43-28b4c9f95aa0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:25:57.151891 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3615cfad-3621-434d-bb43-28b4c9f95aa0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:57.157437 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3615cfad-3621-434d-bb43-28b4c9f95aa0 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:57.161523 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a60b6-394f-4c8d-8d77-223795589831 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:57.170520 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e8a838-860e-4944-be86-409e92c7665b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:57.171722 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8a838-860e-4944-be86-409e92c7665b service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:57.171722 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8a838-860e-4944-be86-409e92c7665b service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:57.171722 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8a838-860e-4944-be86-409e92c7665b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:57.184462 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e8a838-860e-4944-be86-409e92c7665b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:57.184678 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 939/1851] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:25:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:25:57.184764 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e8a838-860e-4944-be86-409e92c7665b service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:57.188666 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3615cfad-3621-434d-bb43-28b4c9f95aa0 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:57.192671 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:57.201061 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 913/1852] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:57.204179 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3615cfad-3621-434d-bb43-28b4c9f95aa0 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:57.214250 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5361d02-f975-41da-9710-add2faf46aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:57.215338 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5361d02-f975-41da-9710-add2faf46aef admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:57.215968 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5361d02-f975-41da-9710-add2faf46aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:57.216650 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5361d02-f975-41da-9710-add2faf46aef admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:57.217966 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5361d02-f975-41da-9710-add2faf46aef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:57.228433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5361d02-f975-41da-9710-add2faf46aef admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:57.355157 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 940/1853] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:25:57 2022] POST /identity/v3/projects => generated 409 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:57.355898 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e8a838-860e-4944-be86-409e92c7665b service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:57.369383 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e8a838-860e-4944-be86-409e92c7665b service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:57.457121 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e00c91e-b46e-4cd7-a796-b44490e18e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:57.460623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00c91e-b46e-4cd7-a796-b44490e18e56 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:57.464903 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00c91e-b46e-4cd7-a796-b44490e18e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:57.465186 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00c91e-b46e-4cd7-a796-b44490e18e56 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:57.466493 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e00c91e-b46e-4cd7-a796-b44490e18e56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:57.468760 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e00c91e-b46e-4cd7-a796-b44490e18e56 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:57.570135 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 914/1854] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:25:57 2022] POST /identity/v3/users => generated 305 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:25:57.575130 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5361d02-f975-41da-9710-add2faf46aef admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:57.607161 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5361d02-f975-41da-9710-add2faf46aef admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:57.619057 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7616a0d-2f09-4f2c-9cd5-6a34ef779be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:57.619840 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7616a0d-2f09-4f2c-9cd5-6a34ef779be1 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:57.620034 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7616a0d-2f09-4f2c-9cd5-6a34ef779be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:57.620279 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7616a0d-2f09-4f2c-9cd5-6a34ef779be1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:57.621774 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7616a0d-2f09-4f2c-9cd5-6a34ef779be1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:57.625647 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7616a0d-2f09-4f2c-9cd5-6a34ef779be1 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:57.646521 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 941/1855] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:57.647284 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e00c91e-b46e-4cd7-a796-b44490e18e56 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:57.668480 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e00c91e-b46e-4cd7-a796-b44490e18e56 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:57.685212 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83544b44-609d-48d3-a366-94815759994e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:57.686341 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83544b44-609d-48d3-a366-94815759994e admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:57.686341 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83544b44-609d-48d3-a366-94815759994e admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:57.686544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83544b44-609d-48d3-a366-94815759994e admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:57.688774 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83544b44-609d-48d3-a366-94815759994e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:57.692616 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83544b44-609d-48d3-a366-94815759994e admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:57.721597 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 915/1856] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:57.725179 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7616a0d-2f09-4f2c-9cd5-6a34ef779be1 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:57.736604 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7616a0d-2f09-4f2c-9cd5-6a34ef779be1 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:57.741367 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0acc29-a8f2-4a58-9fcd-862ccb46f0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:57.742033 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0acc29-a8f2-4a58-9fcd-862ccb46f0d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:57.742316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0acc29-a8f2-4a58-9fcd-862ccb46f0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:57.742576 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0acc29-a8f2-4a58-9fcd-862ccb46f0d2 admin admin] PATH_INFO: `/v3/projects/3b45f8eb1f784c4f87d2d0560997f0ad/users/e6736c8f94254a8897146abedb29593c/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:57.758224 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0acc29-a8f2-4a58-9fcd-862ccb46f0d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b45f8eb1f784c4f87d2d0560997f0ad, user_id=e6736c8f94254a8897146abedb29593c, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:57.761117 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0acc29-a8f2-4a58-9fcd-862ccb46f0d2 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:57.975060 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 942/1857] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:25:57 2022] PUT /identity/v3/projects/3b45f8eb1f784c4f87d2d0560997f0ad/users/e6736c8f94254a8897146abedb29593c/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:57.985595 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7903e8-8ca5-4d58-a9be-6f25a8d2c9ba None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:57.985893 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7903e8-8ca5-4d58-a9be-6f25a8d2c9ba None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:57.986190 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7903e8-8ca5-4d58-a9be-6f25a8d2c9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:58.015697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-aa7903e8-8ca5-4d58-a9be-6f25a8d2c9ba None None] MFA Rules not processed for user `e6736c8f94254a8897146abedb29593c`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:58.017143 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-aa7903e8-8ca5-4d58-a9be-6f25a8d2c9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:58.115844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 916/1858] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:57 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:58.120581 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0acc29-a8f2-4a58-9fcd-862ccb46f0d2 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:58.133304 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7903e8-8ca5-4d58-a9be-6f25a8d2c9ba None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:58.141849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-aa7903e8-8ca5-4d58-a9be-6f25a8d2c9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:58.222775 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0acc29-a8f2-4a58-9fcd-862ccb46f0d2 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:58.228532 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b77d9a-6578-49ae-a866-677b2c83980a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:58.229316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b77d9a-6578-49ae-a866-677b2c83980a admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:58.229614 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b77d9a-6578-49ae-a866-677b2c83980a admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:58.229887 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b77d9a-6578-49ae-a866-677b2c83980a admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:58.230573 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b77d9a-6578-49ae-a866-677b2c83980a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:58.232549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b77d9a-6578-49ae-a866-677b2c83980a admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:58.240974 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7903e8-8ca5-4d58-a9be-6f25a8d2c9ba None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:58.241911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:25:58.247750 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb9f3a-46ee-47b7-9887-2fa6caba3595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:58.248481 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb9f3a-46ee-47b7-9887-2fa6caba3595 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:58.248763 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb9f3a-46ee-47b7-9887-2fa6caba3595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:58.249580 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb9f3a-46ee-47b7-9887-2fa6caba3595 admin admin] PATH_INFO: `/v3/projects/0aad2d43c2ac4da18ecf568e499fd6eb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:58.250849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eb9f3a-46ee-47b7-9887-2fa6caba3595 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0aad2d43c2ac4da18ecf568e499fd6eb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:58.254150 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eb9f3a-46ee-47b7-9887-2fa6caba3595 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:58.358263 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 943/1859] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:25:58 2022] POST /identity/v3/projects => generated 409 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:25:58.362232 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b77d9a-6578-49ae-a866-677b2c83980a admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:58.372387 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e8eb9f3a-46ee-47b7-9887-2fa6caba3595 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0aad2d43c2ac4da18ecf568e499fd6eb', 'request_id': 'req-e8eb9f3a-46ee-47b7-9887-2fa6caba3595'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:58.379493 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e8eb9f3a-46ee-47b7-9887-2fa6caba3595 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0aad2d43c2ac4da18ecf568e499fd6eb', 'request_id': 'req-e8eb9f3a-46ee-47b7-9887-2fa6caba3595'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:58.427718 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b77d9a-6578-49ae-a866-677b2c83980a admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:58.432393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f481299-a6b5-4da5-a3f3-06044489fa06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:58.433182 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f481299-a6b5-4da5-a3f3-06044489fa06 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:58.433482 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f481299-a6b5-4da5-a3f3-06044489fa06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:58.433753 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f481299-a6b5-4da5-a3f3-06044489fa06 admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:58.434414 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f481299-a6b5-4da5-a3f3-06044489fa06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:58.434590 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e8eb9f3a-46ee-47b7-9887-2fa6caba3595 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0aad2d43c2ac4da18ecf568e499fd6eb', 'request_id': 'req-e8eb9f3a-46ee-47b7-9887-2fa6caba3595'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:58.436347 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f481299-a6b5-4da5-a3f3-06044489fa06 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:58.451887 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e8eb9f3a-46ee-47b7-9887-2fa6caba3595 admin admin] The token cache is being invalidate because project 0aad2d43c2ac4da18ecf568e499fd6eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:25:58.452206 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e8eb9f3a-46ee-47b7-9887-2fa6caba3595 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:58.452911 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e8eb9f3a-46ee-47b7-9887-2fa6caba3595 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:58.454440 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 917/1860] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:25:58 2022] DELETE /identity/v3/projects/0aad2d43c2ac4da18ecf568e499fd6eb => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:58.458871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb9f3a-46ee-47b7-9887-2fa6caba3595 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:58.460274 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e8eb9f3a-46ee-47b7-9887-2fa6caba3595 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:58.522146 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 944/1861] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:25:58 2022] POST /identity/v3/users => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:25:58.526438 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f481299-a6b5-4da5-a3f3-06044489fa06 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:58.528275 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-7f481299-a6b5-4da5-a3f3-06044489fa06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:58.529135 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb9f3a-46ee-47b7-9887-2fa6caba3595 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:58.534007 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7ae6b-0bd2-42d4-a9d6-d2a8f6b8fd68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:58.535250 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7ae6b-0bd2-42d4-a9d6-d2a8f6b8fd68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:58.535597 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7ae6b-0bd2-42d4-a9d6-d2a8f6b8fd68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:58.535876 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7ae6b-0bd2-42d4-a9d6-d2a8f6b8fd68 admin admin] PATH_INFO: `/v3/users/68cad796a46f44eb9ed2c7a15cb59e8c` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:58.538719 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d7ae6b-0bd2-42d4-a9d6-d2a8f6b8fd68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68cad796a46f44eb9ed2c7a15cb59e8c)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:58.542002 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d7ae6b-0bd2-42d4-a9d6-d2a8f6b8fd68 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:58.542996 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f481299-a6b5-4da5-a3f3-06044489fa06 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:58.549386 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f37d8-560c-4e34-a40f-17383f8195ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:58.549386 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f37d8-560c-4e34-a40f-17383f8195ae admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:58.549610 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f37d8-560c-4e34-a40f-17383f8195ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:58.549800 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f37d8-560c-4e34-a40f-17383f8195ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:58.550541 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24f37d8-560c-4e34-a40f-17383f8195ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:58.552169 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24f37d8-560c-4e34-a40f-17383f8195ae admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:58.564918 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 945/1862] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:58.569624 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f37d8-560c-4e34-a40f-17383f8195ae admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:58.579351 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f37d8-560c-4e34-a40f-17383f8195ae admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:58.584904 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288863ac-9678-4909-b907-06d778b56aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:58.585555 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288863ac-9678-4909-b907-06d778b56aec admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:58.585833 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288863ac-9678-4909-b907-06d778b56aec admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:58.586104 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288863ac-9678-4909-b907-06d778b56aec admin admin] PATH_INFO: `/v3/projects/27d0699cdab14f7a8fad4c29c40978e4/users/5530d1aca3c84774b95a0c7197cb01ff/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:58.600768 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288863ac-9678-4909-b907-06d778b56aec admin admin] RBAC: Authorizing `identity:create_grant(project_id=27d0699cdab14f7a8fad4c29c40978e4, user_id=5530d1aca3c84774b95a0c7197cb01ff, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:58.603444 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288863ac-9678-4909-b907-06d778b56aec admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:58.712554 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 946/1863] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:25:58 2022] PUT /identity/v3/projects/27d0699cdab14f7a8fad4c29c40978e4/users/5530d1aca3c84774b95a0c7197cb01ff/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:25:58.722083 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288863ac-9678-4909-b907-06d778b56aec admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:58.741936 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-15d7ae6b-0bd2-42d4-a9d6-d2a8f6b8fd68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68cad796a46f44eb9ed2c7a15cb59e8c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:58.758840 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-15d7ae6b-0bd2-42d4-a9d6-d2a8f6b8fd68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68cad796a46f44eb9ed2c7a15cb59e8c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:58.794375 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288863ac-9678-4909-b907-06d778b56aec admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:58.799048 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe23cb7-8957-4471-896e-8536295023c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:58.799655 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe23cb7-8957-4471-896e-8536295023c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:58.799947 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe23cb7-8957-4471-896e-8536295023c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:58.800241 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe23cb7-8957-4471-896e-8536295023c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:58.800967 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe23cb7-8957-4471-896e-8536295023c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:58.802553 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe23cb7-8957-4471-896e-8536295023c0 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:58.812672 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 947/1864] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:25:58.817308 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe23cb7-8957-4471-896e-8536295023c0 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:58.827089 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe23cb7-8957-4471-896e-8536295023c0 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:58.832375 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4828382-d0f3-4a68-a2c8-32157edf2cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:58.833054 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4828382-d0f3-4a68-a2c8-32157edf2cda admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:58.833336 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4828382-d0f3-4a68-a2c8-32157edf2cda admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:58.833616 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4828382-d0f3-4a68-a2c8-32157edf2cda admin admin] PATH_INFO: `/v3/projects/27d0699cdab14f7a8fad4c29c40978e4/users/5530d1aca3c84774b95a0c7197cb01ff/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:58.835574 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4828382-d0f3-4a68-a2c8-32157edf2cda admin admin] RBAC: Authorizing `identity:create_grant(project_id=27d0699cdab14f7a8fad4c29c40978e4, user_id=5530d1aca3c84774b95a0c7197cb01ff, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:58.838240 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4828382-d0f3-4a68-a2c8-32157edf2cda admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:58.848203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-15d7ae6b-0bd2-42d4-a9d6-d2a8f6b8fd68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68cad796a46f44eb9ed2c7a15cb59e8c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:58.848203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-15d7ae6b-0bd2-42d4-a9d6-d2a8f6b8fd68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68cad796a46f44eb9ed2c7a15cb59e8c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:58.854383 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 948/1865] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:25:58 2022] PUT /identity/v3/projects/27d0699cdab14f7a8fad4c29c40978e4/users/5530d1aca3c84774b95a0c7197cb01ff/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:25:58.856100 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-15d7ae6b-0bd2-42d4-a9d6-d2a8f6b8fd68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68cad796a46f44eb9ed2c7a15cb59e8c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:58.863163 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 918/1866] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:25:58 2022] DELETE /identity/v3/users/68cad796a46f44eb9ed2c7a15cb59e8c => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:25:58.870015 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26705372-d2b8-4616-b16b-0d84baa7f2d0 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:58.870338 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26705372-d2b8-4616-b16b-0d84baa7f2d0 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:58.870618 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26705372-d2b8-4616-b16b-0d84baa7f2d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:58.905224 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-26705372-d2b8-4616-b16b-0d84baa7f2d0 None None] MFA Rules not processed for user `5530d1aca3c84774b95a0c7197cb01ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:25:58.906685 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-26705372-d2b8-4616-b16b-0d84baa7f2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:59.025995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 949/1867] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:25:58 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:25:59.042879 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7ae6b-0bd2-42d4-a9d6-d2a8f6b8fd68 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:59.044405 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-15d7ae6b-0bd2-42d4-a9d6-d2a8f6b8fd68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:59.105032 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7ae6b-0bd2-42d4-a9d6-d2a8f6b8fd68 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:59.108661 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0fb21-c399-4b14-97a0-b8f31d298fcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:59.109334 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0fb21-c399-4b14-97a0-b8f31d298fcc service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:59.109620 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0fb21-c399-4b14-97a0-b8f31d298fcc service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:59.109894 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0fb21-c399-4b14-97a0-b8f31d298fcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:59.120773 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db0fb21-c399-4b14-97a0-b8f31d298fcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:59.122361 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db0fb21-c399-4b14-97a0-b8f31d298fcc service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:59.132572 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 919/1868] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:25:59 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:25:59.559856 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26705372-d2b8-4616-b16b-0d84baa7f2d0 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:59.561611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-26705372-d2b8-4616-b16b-0d84baa7f2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:59.579638 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0fb21-c399-4b14-97a0-b8f31d298fcc service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:25:59.581910 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-1db0fb21-c399-4b14-97a0-b8f31d298fcc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:25:59.667805 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26705372-d2b8-4616-b16b-0d84baa7f2d0 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:59.672857 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819d574-e6b9-486b-bf80-461847e5f6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:59.673667 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819d574-e6b9-486b-bf80-461847e5f6b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:59.673993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819d574-e6b9-486b-bf80-461847e5f6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:59.674263 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819d574-e6b9-486b-bf80-461847e5f6b4 admin admin] PATH_INFO: `/v3/users/79a18e6676da4c6088ed9bd79843500a` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:59.675844 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9819d574-e6b9-486b-bf80-461847e5f6b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79a18e6676da4c6088ed9bd79843500a)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:59.678256 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9819d574-e6b9-486b-bf80-461847e5f6b4 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:59.687146 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0fb21-c399-4b14-97a0-b8f31d298fcc service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:25:59.692179 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1151ffb-b86e-459d-ae94-23fef134aa0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:25:59.692973 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1151ffb-b86e-459d-ae94-23fef134aa0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:25:59.693346 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1151ffb-b86e-459d-ae94-23fef134aa0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:25:59.693695 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1151ffb-b86e-459d-ae94-23fef134aa0c admin admin] PATH_INFO: `/v3/projects/a3d9591db1f8459083c1c9ff5f316f63` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:25:59.695030 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1151ffb-b86e-459d-ae94-23fef134aa0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3d9591db1f8459083c1c9ff5f316f63)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:25:59.697063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1151ffb-b86e-459d-ae94-23fef134aa0c admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:25:59.916166 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9819d574-e6b9-486b-bf80-461847e5f6b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79a18e6676da4c6088ed9bd79843500a'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:59.916500 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e1151ffb-b86e-459d-ae94-23fef134aa0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3d9591db1f8459083c1c9ff5f316f63', 'request_id': 'req-e1151ffb-b86e-459d-ae94-23fef134aa0c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:59.922637 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e1151ffb-b86e-459d-ae94-23fef134aa0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3d9591db1f8459083c1c9ff5f316f63', 'request_id': 'req-e1151ffb-b86e-459d-ae94-23fef134aa0c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:59.923378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9819d574-e6b9-486b-bf80-461847e5f6b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79a18e6676da4c6088ed9bd79843500a'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:59.927908 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9819d574-e6b9-486b-bf80-461847e5f6b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79a18e6676da4c6088ed9bd79843500a'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:59.927908 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9819d574-e6b9-486b-bf80-461847e5f6b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79a18e6676da4c6088ed9bd79843500a'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:25:59.936867 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9819d574-e6b9-486b-bf80-461847e5f6b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79a18e6676da4c6088ed9bd79843500a'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:00.017604 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e1151ffb-b86e-459d-ae94-23fef134aa0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3d9591db1f8459083c1c9ff5f316f63', 'request_id': 'req-e1151ffb-b86e-459d-ae94-23fef134aa0c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:00.028918 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e1151ffb-b86e-459d-ae94-23fef134aa0c admin admin] The token cache is being invalidate because project a3d9591db1f8459083c1c9ff5f316f63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:26:00.029298 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e1151ffb-b86e-459d-ae94-23fef134aa0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:00.029920 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e1151ffb-b86e-459d-ae94-23fef134aa0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:00.031546 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 920/1869] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:25:59 2022] DELETE /identity/v3/projects/a3d9591db1f8459083c1c9ff5f316f63 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:00.035589 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1151ffb-b86e-459d-ae94-23fef134aa0c admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:00.035589 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e1151ffb-b86e-459d-ae94-23fef134aa0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:00.091404 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1151ffb-b86e-459d-ae94-23fef134aa0c admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:00.095712 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e197815-4db3-4862-b32a-cf43ae8397d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:00.096320 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e197815-4db3-4862-b32a-cf43ae8397d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:00.096651 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e197815-4db3-4862-b32a-cf43ae8397d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:00.097069 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e197815-4db3-4862-b32a-cf43ae8397d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:00.098925 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5e197815-4db3-4862-b32a-cf43ae8397d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:00.136852 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 950/1870] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:25:59 2022] DELETE /identity/v3/users/79a18e6676da4c6088ed9bd79843500a => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:00.138194 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2849dcd-ed85-42b2-9763-688bc86cdd1f None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:00.138687 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2849dcd-ed85-42b2-9763-688bc86cdd1f None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:00.138805 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2849dcd-ed85-42b2-9763-688bc86cdd1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:00.159483 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:00.161923 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-e2849dcd-ed85-42b2-9763-688bc86cdd1f None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:00.162947 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e197815-4db3-4862-b32a-cf43ae8397d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:00.163208 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-e2849dcd-ed85-42b2-9763-688bc86cdd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:00.164384 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e197815-4db3-4862-b32a-cf43ae8397d3 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:00.173531 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 921/1871] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:00 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:26:00.175052 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e18098-9a70-4678-83de-45e16744ca5f None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:00.175180 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e18098-9a70-4678-83de-45e16744ca5f None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:00.175633 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e18098-9a70-4678-83de-45e16744ca5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:00.194708 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-72e18098-9a70-4678-83de-45e16744ca5f None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:00.195920 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-72e18098-9a70-4678-83de-45e16744ca5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:00.234293 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 951/1872] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:00.235855 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d53d5-e12c-4d61-a52b-6309b13b9485 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:00.236150 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d53d5-e12c-4d61-a52b-6309b13b9485 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:00.236450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d53d5-e12c-4d61-a52b-6309b13b9485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:00.267573 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-e29d53d5-e12c-4d61-a52b-6309b13b9485 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:00.269323 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 922/1873] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:00.270075 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-e29d53d5-e12c-4d61-a52b-6309b13b9485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:00.279155 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 952/1874] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:26:00.280574 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e18098-9a70-4678-83de-45e16744ca5f None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:00.282069 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-72e18098-9a70-4678-83de-45e16744ca5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:00.284073 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d53d5-e12c-4d61-a52b-6309b13b9485 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:00.295446 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d53d5-e12c-4d61-a52b-6309b13b9485 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:00.302618 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cddbfac-37d5-4ae4-a92f-a509874dcaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:00.303812 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cddbfac-37d5-4ae4-a92f-a509874dcaec admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:00.303812 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cddbfac-37d5-4ae4-a92f-a509874dcaec admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:00.304020 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cddbfac-37d5-4ae4-a92f-a509874dcaec admin admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:00.305060 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cddbfac-37d5-4ae4-a92f-a509874dcaec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:26:00.305316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cddbfac-37d5-4ae4-a92f-a509874dcaec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:00.307224 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cddbfac-37d5-4ae4-a92f-a509874dcaec admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:00.326681 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:00.328438 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 953/1875] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:26:00.333690 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cddbfac-37d5-4ae4-a92f-a509874dcaec admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:00.347401 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cddbfac-37d5-4ae4-a92f-a509874dcaec admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:00.353572 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c9bbf-4db6-40fa-8196-fd34e7a13956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:00.354235 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c9bbf-4db6-40fa-8196-fd34e7a13956 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:00.354518 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c9bbf-4db6-40fa-8196-fd34e7a13956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:00.354793 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c9bbf-4db6-40fa-8196-fd34e7a13956 admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:00.355737 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426c9bbf-4db6-40fa-8196-fd34e7a13956 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:00.358451 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426c9bbf-4db6-40fa-8196-fd34e7a13956 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:00.368685 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e18098-9a70-4678-83de-45e16744ca5f None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:00.374417 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2fdec-5b16-4588-8f1a-063d33ab180f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:00.376242 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2fdec-5b16-4588-8f1a-063d33ab180f service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:00.376564 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2fdec-5b16-4588-8f1a-063d33ab180f service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:00.376841 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2fdec-5b16-4588-8f1a-063d33ab180f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:00.391568 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef2fdec-5b16-4588-8f1a-063d33ab180f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:00.393049 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef2fdec-5b16-4588-8f1a-063d33ab180f service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:00.403885 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 923/1876] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:26:00.506203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2fdec-5b16-4588-8f1a-063d33ab180f service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:00.506516 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 954/1877] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:26:00 2022] POST /identity/v3/projects => generated 383 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:00.532306 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2fdec-5b16-4588-8f1a-063d33ab180f service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:00.590623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17162c4d-94fc-4bc2-a1bc-a1c145186bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:00.591273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17162c4d-94fc-4bc2-a1bc-a1c145186bc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:00.591509 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17162c4d-94fc-4bc2-a1bc-a1c145186bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:00.591783 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17162c4d-94fc-4bc2-a1bc-a1c145186bc3 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:00.592749 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17162c4d-94fc-4bc2-a1bc-a1c145186bc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:00.594670 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17162c4d-94fc-4bc2-a1bc-a1c145186bc3 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:00.701856 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 924/1878] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:26:00 2022] POST /identity/v3/users => generated 292 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:00.704695 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c9bbf-4db6-40fa-8196-fd34e7a13956 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:00.720807 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c9bbf-4db6-40fa-8196-fd34e7a13956 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:00.728571 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e13bc-4055-459c-a704-e8307c177136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:00.728571 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e13bc-4055-459c-a704-e8307c177136 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:00.729599 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e13bc-4055-459c-a704-e8307c177136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:00.729874 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e13bc-4055-459c-a704-e8307c177136 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:00.730744 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43e13bc-4055-459c-a704-e8307c177136 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:00.732399 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43e13bc-4055-459c-a704-e8307c177136 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:00.746258 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 955/1879] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:26:00.747885 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17162c4d-94fc-4bc2-a1bc-a1c145186bc3 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:00.757809 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17162c4d-94fc-4bc2-a1bc-a1c145186bc3 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:00.762877 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17b9d9-6c09-47cf-bf71-207246f1f456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:00.763496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17b9d9-6c09-47cf-bf71-207246f1f456 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:00.763856 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17b9d9-6c09-47cf-bf71-207246f1f456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:00.764130 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17b9d9-6c09-47cf-bf71-207246f1f456 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:00.764905 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a17b9d9-6c09-47cf-bf71-207246f1f456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:00.767090 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a17b9d9-6c09-47cf-bf71-207246f1f456 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:00.779370 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 925/1880] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:26:00.783547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e13bc-4055-459c-a704-e8307c177136 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:00.797190 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e13bc-4055-459c-a704-e8307c177136 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:00.804082 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10766f7b-5929-4678-9b00-139334daae75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:00.804982 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10766f7b-5929-4678-9b00-139334daae75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:00.811334 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10766f7b-5929-4678-9b00-139334daae75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:00.811791 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10766f7b-5929-4678-9b00-139334daae75 admin admin] PATH_INFO: `/v3/projects/567dbab3a66a489aa09e15c582f73c9f/users/4cfe2d28f8e94d63a334ec53c100e889/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:00.832865 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10766f7b-5929-4678-9b00-139334daae75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=567dbab3a66a489aa09e15c582f73c9f, user_id=4cfe2d28f8e94d63a334ec53c100e889, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:00.835657 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10766f7b-5929-4678-9b00-139334daae75 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:00.944838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 956/1881] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:26:00 2022] PUT /identity/v3/projects/567dbab3a66a489aa09e15c582f73c9f/users/4cfe2d28f8e94d63a334ec53c100e889/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:26:00.954716 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21aa83-9cea-46e1-b1d0-812090f2b292 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:00.954998 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21aa83-9cea-46e1-b1d0-812090f2b292 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:00.955266 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21aa83-9cea-46e1-b1d0-812090f2b292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:00.987529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-ce21aa83-9cea-46e1-b1d0-812090f2b292 None None] MFA Rules not processed for user `4cfe2d28f8e94d63a334ec53c100e889`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:00.988793 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ce21aa83-9cea-46e1-b1d0-812090f2b292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:01.045031 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10766f7b-5929-4678-9b00-139334daae75 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:01.045031 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-10766f7b-5929-4678-9b00-139334daae75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:01.073436 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 926/1882] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:01.077514 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce21aa83-9cea-46e1-b1d0-812090f2b292 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:01.139832 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10766f7b-5929-4678-9b00-139334daae75 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:01.145729 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b962233-50fe-47a9-af2b-a53ea39847d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:01.146512 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b962233-50fe-47a9-af2b-a53ea39847d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:01.146903 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b962233-50fe-47a9-af2b-a53ea39847d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:01.147241 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b962233-50fe-47a9-af2b-a53ea39847d8 admin admin] PATH_INFO: `/v3/projects/8ee5da8897ed438f9a0304baf9ace198` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:01.148499 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b962233-50fe-47a9-af2b-a53ea39847d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ee5da8897ed438f9a0304baf9ace198)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:01.150582 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b962233-50fe-47a9-af2b-a53ea39847d8 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:01.157306 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce21aa83-9cea-46e1-b1d0-812090f2b292 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:01.163208 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1826641-4e29-4aae-a9f5-06e6ffc26cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:01.163880 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1826641-4e29-4aae-a9f5-06e6ffc26cb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:01.164165 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1826641-4e29-4aae-a9f5-06e6ffc26cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:01.168760 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1826641-4e29-4aae-a9f5-06e6ffc26cb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:01.168760 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1826641-4e29-4aae-a9f5-06e6ffc26cb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:01.168760 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1826641-4e29-4aae-a9f5-06e6ffc26cb2 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:01.275747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-1b962233-50fe-47a9-af2b-a53ea39847d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ee5da8897ed438f9a0304baf9ace198', 'request_id': 'req-1b962233-50fe-47a9-af2b-a53ea39847d8'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:01.281759 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-1b962233-50fe-47a9-af2b-a53ea39847d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ee5da8897ed438f9a0304baf9ace198', 'request_id': 'req-1b962233-50fe-47a9-af2b-a53ea39847d8'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:01.287771 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-1b962233-50fe-47a9-af2b-a53ea39847d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ee5da8897ed438f9a0304baf9ace198', 'request_id': 'req-1b962233-50fe-47a9-af2b-a53ea39847d8'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:01.383071 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-1b962233-50fe-47a9-af2b-a53ea39847d8 admin admin] The token cache is being invalidate because project 8ee5da8897ed438f9a0304baf9ace198 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:26:01.383468 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-1b962233-50fe-47a9-af2b-a53ea39847d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:01.384144 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-1b962233-50fe-47a9-af2b-a53ea39847d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:01.389118 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 927/1884] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:26:01 2022] POST /identity/v3/projects => generated 383 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:26:01.389118 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1826641-4e29-4aae-a9f5-06e6ffc26cb2 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:01.389493 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 957/1884] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:26:01 2022] DELETE /identity/v3/projects/8ee5da8897ed438f9a0304baf9ace198 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:01.389906 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e1826641-4e29-4aae-a9f5-06e6ffc26cb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:01.390019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b962233-50fe-47a9-af2b-a53ea39847d8 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:01.391383 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-1b962233-50fe-47a9-af2b-a53ea39847d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:01.453417 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1826641-4e29-4aae-a9f5-06e6ffc26cb2 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:01.454007 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b962233-50fe-47a9-af2b-a53ea39847d8 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:01.458515 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e9513-b3dc-4c27-ae02-83b1a1cd9447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:01.459147 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e9513-b3dc-4c27-ae02-83b1a1cd9447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:01.459271 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14f540d-c8a6-485d-82d9-23d1265927d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:01.459421 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e9513-b3dc-4c27-ae02-83b1a1cd9447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:01.459689 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e9513-b3dc-4c27-ae02-83b1a1cd9447 admin admin] PATH_INFO: `/v3/users/24f59721b45943378ca6fcc9d3a5723c` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:01.459903 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f540d-c8a6-485d-82d9-23d1265927d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:01.460187 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f540d-c8a6-485d-82d9-23d1265927d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:01.460484 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f540d-c8a6-485d-82d9-23d1265927d8 admin admin] PATH_INFO: `/v3/users/17e59984bc7145cebe0c0ab15f68549a` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:01.460860 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e9513-b3dc-4c27-ae02-83b1a1cd9447 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24f59721b45943378ca6fcc9d3a5723c)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:01.461689 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14f540d-c8a6-485d-82d9-23d1265927d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17e59984bc7145cebe0c0ab15f68549a)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:01.463000 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e9513-b3dc-4c27-ae02-83b1a1cd9447 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:01.463575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14f540d-c8a6-485d-82d9-23d1265927d8 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:01.672326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-b51e9513-b3dc-4c27-ae02-83b1a1cd9447 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24f59721b45943378ca6fcc9d3a5723c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:01.674087 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-d14f540d-c8a6-485d-82d9-23d1265927d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17e59984bc7145cebe0c0ab15f68549a'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:01.682293 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-d14f540d-c8a6-485d-82d9-23d1265927d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17e59984bc7145cebe0c0ab15f68549a'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:01.683131 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-d14f540d-c8a6-485d-82d9-23d1265927d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17e59984bc7145cebe0c0ab15f68549a'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:01.683803 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-d14f540d-c8a6-485d-82d9-23d1265927d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17e59984bc7145cebe0c0ab15f68549a'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:01.686210 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-b51e9513-b3dc-4c27-ae02-83b1a1cd9447 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24f59721b45943378ca6fcc9d3a5723c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:01.697566 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-d14f540d-c8a6-485d-82d9-23d1265927d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17e59984bc7145cebe0c0ab15f68549a'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:01.783950 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-b51e9513-b3dc-4c27-ae02-83b1a1cd9447 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24f59721b45943378ca6fcc9d3a5723c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:01.784677 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-b51e9513-b3dc-4c27-ae02-83b1a1cd9447 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24f59721b45943378ca6fcc9d3a5723c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:01.792488 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-b51e9513-b3dc-4c27-ae02-83b1a1cd9447 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24f59721b45943378ca6fcc9d3a5723c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:01.798800 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 928/1885] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:26:01 2022] DELETE /identity/v3/users/24f59721b45943378ca6fcc9d3a5723c => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:01.799634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e9513-b3dc-4c27-ae02-83b1a1cd9447 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:01.801201 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-b51e9513-b3dc-4c27-ae02-83b1a1cd9447 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:01.851029 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 958/1886] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:26:01 2022] DELETE /identity/v3/users/17e59984bc7145cebe0c0ab15f68549a => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:26:01.853198 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff48055-ab72-4356-8e83-c8d43a5526c6 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:01.853726 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff48055-ab72-4356-8e83-c8d43a5526c6 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:01.854132 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff48055-ab72-4356-8e83-c8d43a5526c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:01.859560 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:01.881238 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-4ff48055-ab72-4356-8e83-c8d43a5526c6 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:01.882884 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-4ff48055-ab72-4356-8e83-c8d43a5526c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:01.904019 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e9513-b3dc-4c27-ae02-83b1a1cd9447 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:01.975802 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c071631d-04bf-4ec7-be57-4775301ffc53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:01.976548 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c071631d-04bf-4ec7-be57-4775301ffc53 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:01.976925 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c071631d-04bf-4ec7-be57-4775301ffc53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:01.977297 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c071631d-04bf-4ec7-be57-4775301ffc53 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:01.978153 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c071631d-04bf-4ec7-be57-4775301ffc53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:01.980316 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c071631d-04bf-4ec7-be57-4775301ffc53 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:01.983288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 959/1887] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:01.986968 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:01.994210 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff48055-ab72-4356-8e83-c8d43a5526c6 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:01.995802 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-4ff48055-ab72-4356-8e83-c8d43a5526c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:02.062946 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff48055-ab72-4356-8e83-c8d43a5526c6 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:02.067652 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5a80fa-8616-4d33-80af-50c5bfe3166a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:02.068327 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a80fa-8616-4d33-80af-50c5bfe3166a service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:02.068644 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a80fa-8616-4d33-80af-50c5bfe3166a service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:02.068915 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a80fa-8616-4d33-80af-50c5bfe3166a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:02.081998 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5a80fa-8616-4d33-80af-50c5bfe3166a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:02.083684 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5a80fa-8616-4d33-80af-50c5bfe3166a service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:02.094486 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 960/1888] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 14:26:02.104960 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 929/1889] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:26:01 2022] POST /identity/v3/users => generated 298 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:02.110764 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5a80fa-8616-4d33-80af-50c5bfe3166a service neutron] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:02.124500 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5a80fa-8616-4d33-80af-50c5bfe3166a service neutron] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:02.134283 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a994368a-7597-433c-a283-271ab5d7396e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:02.135563 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994368a-7597-433c-a283-271ab5d7396e admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:02.135992 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994368a-7597-433c-a283-271ab5d7396e admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:02.136477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994368a-7597-433c-a283-271ab5d7396e admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:02.137331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a994368a-7597-433c-a283-271ab5d7396e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:02.139190 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a994368a-7597-433c-a283-271ab5d7396e admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:02.163733 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 961/1890] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:26:02.165175 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c071631d-04bf-4ec7-be57-4775301ffc53 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:02.177002 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c071631d-04bf-4ec7-be57-4775301ffc53 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:02.181787 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2edec05-a390-450f-87f8-a8863e17a8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:02.182518 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2edec05-a390-450f-87f8-a8863e17a8a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:02.182877 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2edec05-a390-450f-87f8-a8863e17a8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:02.183209 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2edec05-a390-450f-87f8-a8863e17a8a4 admin admin] PATH_INFO: `/v3/projects/01ddb9a36c2b4d7786644b4c04efc1e5/users/561ec5349565401fb2f758af3314b755/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:02.200095 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2edec05-a390-450f-87f8-a8863e17a8a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01ddb9a36c2b4d7786644b4c04efc1e5, user_id=561ec5349565401fb2f758af3314b755, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:02.202805 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2edec05-a390-450f-87f8-a8863e17a8a4 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:02.309610 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 930/1891] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:26:02 2022] PUT /identity/v3/projects/01ddb9a36c2b4d7786644b4c04efc1e5/users/561ec5349565401fb2f758af3314b755/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:02.313617 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a994368a-7597-433c-a283-271ab5d7396e admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:02.323832 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a994368a-7597-433c-a283-271ab5d7396e admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:02.382022 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ad51d-5a1f-4202-a385-75b2515a5600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:02.382646 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ad51d-5a1f-4202-a385-75b2515a5600 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:02.382926 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ad51d-5a1f-4202-a385-75b2515a5600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:02.383197 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ad51d-5a1f-4202-a385-75b2515a5600 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:02.384220 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42ad51d-5a1f-4202-a385-75b2515a5600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:02.385486 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42ad51d-5a1f-4202-a385-75b2515a5600 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:02.397671 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 962/1892] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:26:02.401976 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2edec05-a390-450f-87f8-a8863e17a8a4 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:02.420640 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2edec05-a390-450f-87f8-a8863e17a8a4 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:02.426402 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4687ea-116d-4f4b-8ec0-30c4d7dfb119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:02.427110 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4687ea-116d-4f4b-8ec0-30c4d7dfb119 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:02.427394 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4687ea-116d-4f4b-8ec0-30c4d7dfb119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:02.427668 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4687ea-116d-4f4b-8ec0-30c4d7dfb119 admin admin] PATH_INFO: `/v3/projects/01ddb9a36c2b4d7786644b4c04efc1e5/users/561ec5349565401fb2f758af3314b755/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:02.429729 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4687ea-116d-4f4b-8ec0-30c4d7dfb119 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01ddb9a36c2b4d7786644b4c04efc1e5, user_id=561ec5349565401fb2f758af3314b755, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:02.432534 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4687ea-116d-4f4b-8ec0-30c4d7dfb119 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:02.436023 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ad51d-5a1f-4202-a385-75b2515a5600 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:02.437796 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-d42ad51d-5a1f-4202-a385-75b2515a5600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:02.459324 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 931/1893] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:26:02 2022] PUT /identity/v3/projects/01ddb9a36c2b4d7786644b4c04efc1e5/users/561ec5349565401fb2f758af3314b755/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:02.466382 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ad51d-5a1f-4202-a385-75b2515a5600 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:02.471410 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349510fa-652c-4bab-8c54-60dbc4c958fd None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:02.471778 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349510fa-652c-4bab-8c54-60dbc4c958fd None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:02.471968 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349510fa-652c-4bab-8c54-60dbc4c958fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:02.503655 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-349510fa-652c-4bab-8c54-60dbc4c958fd None None] MFA Rules not processed for user `561ec5349565401fb2f758af3314b755`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:02.505006 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-349510fa-652c-4bab-8c54-60dbc4c958fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:02.548541 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca33456-a5b0-4804-8ab0-e5c8b05e8424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:02.549196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca33456-a5b0-4804-8ab0-e5c8b05e8424 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:02.549473 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca33456-a5b0-4804-8ab0-e5c8b05e8424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:02.549742 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca33456-a5b0-4804-8ab0-e5c8b05e8424 admin admin] PATH_INFO: `/v3/projects/f8c178e5cd7c4a9293f9689b8a7c1161` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:02.551282 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca33456-a5b0-4804-8ab0-e5c8b05e8424 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8c178e5cd7c4a9293f9689b8a7c1161)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:02.553249 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca33456-a5b0-4804-8ab0-e5c8b05e8424 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:02.580327 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 932/1894] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:02 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:02.598410 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349510fa-652c-4bab-8c54-60dbc4c958fd None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:02.599840 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-349510fa-652c-4bab-8c54-60dbc4c958fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:02.655324 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349510fa-652c-4bab-8c54-60dbc4c958fd None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:02.659213 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c42dcb-89f4-4d40-9154-793d2bc0fc90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:02.660539 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c42dcb-89f4-4d40-9154-793d2bc0fc90 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:02.660979 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c42dcb-89f4-4d40-9154-793d2bc0fc90 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:02.661323 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c42dcb-89f4-4d40-9154-793d2bc0fc90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:02.670852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c42dcb-89f4-4d40-9154-793d2bc0fc90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:02.672375 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c42dcb-89f4-4d40-9154-793d2bc0fc90 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:02.684635 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 933/1895] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:02 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:26:02.702926 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-7ca33456-a5b0-4804-8ab0-e5c8b05e8424 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8c178e5cd7c4a9293f9689b8a7c1161', 'request_id': 'req-7ca33456-a5b0-4804-8ab0-e5c8b05e8424'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:02.703917 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c42dcb-89f4-4d40-9154-793d2bc0fc90 service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:02.708532 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-99c42dcb-89f4-4d40-9154-793d2bc0fc90 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:02.716521 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-7ca33456-a5b0-4804-8ab0-e5c8b05e8424 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8c178e5cd7c4a9293f9689b8a7c1161', 'request_id': 'req-7ca33456-a5b0-4804-8ab0-e5c8b05e8424'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:02.723652 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-7ca33456-a5b0-4804-8ab0-e5c8b05e8424 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8c178e5cd7c4a9293f9689b8a7c1161', 'request_id': 'req-7ca33456-a5b0-4804-8ab0-e5c8b05e8424'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:02.734293 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c42dcb-89f4-4d40-9154-793d2bc0fc90 service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:02.822404 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e334b2-a3ab-43c5-b904-fb9b27cd6370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:02.824341 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e334b2-a3ab-43c5-b904-fb9b27cd6370 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:02.824978 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e334b2-a3ab-43c5-b904-fb9b27cd6370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:02.825316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-7ca33456-a5b0-4804-8ab0-e5c8b05e8424 admin admin] The token cache is being invalidate because project f8c178e5cd7c4a9293f9689b8a7c1161 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:26:02.825435 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e334b2-a3ab-43c5-b904-fb9b27cd6370 admin admin] PATH_INFO: `/v3/projects/d968144847784badba4ad35e55c20777` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:02.825895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-7ca33456-a5b0-4804-8ab0-e5c8b05e8424 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:02.826836 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e334b2-a3ab-43c5-b904-fb9b27cd6370 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d968144847784badba4ad35e55c20777)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:02.827937 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-7ca33456-a5b0-4804-8ab0-e5c8b05e8424 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:02.829900 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e334b2-a3ab-43c5-b904-fb9b27cd6370 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:02.833795 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 963/1896] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:26:02 2022] DELETE /identity/v3/projects/f8c178e5cd7c4a9293f9689b8a7c1161 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:26:02.836033 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca33456-a5b0-4804-8ab0-e5c8b05e8424 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:02.842534 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-7ca33456-a5b0-4804-8ab0-e5c8b05e8424 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:02.905194 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca33456-a5b0-4804-8ab0-e5c8b05e8424 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:02.908955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da01a1-6066-45b5-884f-969371433d63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:02.909643 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da01a1-6066-45b5-884f-969371433d63 service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:02.910131 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da01a1-6066-45b5-884f-969371433d63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:02.910565 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da01a1-6066-45b5-884f-969371433d63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:02.912338 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-c5da01a1-6066-45b5-884f-969371433d63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:02.936052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:02.959592 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-33e334b2-a3ab-43c5-b904-fb9b27cd6370 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd968144847784badba4ad35e55c20777', 'request_id': 'req-33e334b2-a3ab-43c5-b904-fb9b27cd6370'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:02.973469 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-33e334b2-a3ab-43c5-b904-fb9b27cd6370 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd968144847784badba4ad35e55c20777', 'request_id': 'req-33e334b2-a3ab-43c5-b904-fb9b27cd6370'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:02.990379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da01a1-6066-45b5-884f-969371433d63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:02.991589 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da01a1-6066-45b5-884f-969371433d63 service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:03.001563 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 964/1897] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:26:03.004236 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f180f-23d5-4616-8823-25110c3930f8 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:03.004873 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f180f-23d5-4616-8823-25110c3930f8 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:03.007177 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f180f-23d5-4616-8823-25110c3930f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:03.027419 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-7c3f180f-23d5-4616-8823-25110c3930f8 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:03.028689 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-7c3f180f-23d5-4616-8823-25110c3930f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:03.074487 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-33e334b2-a3ab-43c5-b904-fb9b27cd6370 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd968144847784badba4ad35e55c20777', 'request_id': 'req-33e334b2-a3ab-43c5-b904-fb9b27cd6370'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:03.089053 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-33e334b2-a3ab-43c5-b904-fb9b27cd6370 admin admin] The token cache is being invalidate because project d968144847784badba4ad35e55c20777 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:26:03.089550 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-33e334b2-a3ab-43c5-b904-fb9b27cd6370 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:03.090403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-33e334b2-a3ab-43c5-b904-fb9b27cd6370 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:03.093125 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 934/1898] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:26:02 2022] DELETE /identity/v3/projects/d968144847784badba4ad35e55c20777 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:03.098071 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e334b2-a3ab-43c5-b904-fb9b27cd6370 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:03.099982 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-33e334b2-a3ab-43c5-b904-fb9b27cd6370 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:03.104672 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 965/1899] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:03.116764 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89f192-92aa-4f8d-88ae-ae7d3a55cd3f None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:03.117270 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e334b2-a3ab-43c5-b904-fb9b27cd6370 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:03.117604 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89f192-92aa-4f8d-88ae-ae7d3a55cd3f None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:03.118051 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89f192-92aa-4f8d-88ae-ae7d3a55cd3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:03.122529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d49b169-9cb8-4fd5-9f9f-e91d0a295fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:03.123326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49b169-9cb8-4fd5-9f9f-e91d0a295fe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:03.123801 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49b169-9cb8-4fd5-9f9f-e91d0a295fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:03.124232 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49b169-9cb8-4fd5-9f9f-e91d0a295fe9 admin admin] PATH_INFO: `/v3/users/429b5bf1eccd442a8aa5f561440de79a` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:03.125856 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d49b169-9cb8-4fd5-9f9f-e91d0a295fe9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=429b5bf1eccd442a8aa5f561440de79a)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:03.128949 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d49b169-9cb8-4fd5-9f9f-e91d0a295fe9 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:03.140626 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-bc89f192-92aa-4f8d-88ae-ae7d3a55cd3f None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:03.142145 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-bc89f192-92aa-4f8d-88ae-ae7d3a55cd3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:03.161372 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 966/1900] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:26:03.168690 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc89f192-92aa-4f8d-88ae-ae7d3a55cd3f None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:03.188497 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc89f192-92aa-4f8d-88ae-ae7d3a55cd3f None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:03.196604 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de3ae16-e2e9-4f72-8a0a-9f2ea88d4340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:03.197377 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de3ae16-e2e9-4f72-8a0a-9f2ea88d4340 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:03.197757 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de3ae16-e2e9-4f72-8a0a-9f2ea88d4340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:03.198126 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de3ae16-e2e9-4f72-8a0a-9f2ea88d4340 admin admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:03.199088 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de3ae16-e2e9-4f72-8a0a-9f2ea88d4340 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:26:03.199487 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de3ae16-e2e9-4f72-8a0a-9f2ea88d4340 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:03.202031 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de3ae16-e2e9-4f72-8a0a-9f2ea88d4340 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:03.225546 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 967/1901] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:26:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:26:03.226259 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de3ae16-e2e9-4f72-8a0a-9f2ea88d4340 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:03.237161 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de3ae16-e2e9-4f72-8a0a-9f2ea88d4340 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:03.243426 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c98a5-df34-426e-8b84-9f077fb075ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:03.244278 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c98a5-df34-426e-8b84-9f077fb075ce admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:03.244808 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c98a5-df34-426e-8b84-9f077fb075ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:03.245264 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c98a5-df34-426e-8b84-9f077fb075ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:03.246108 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585c98a5-df34-426e-8b84-9f077fb075ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:03.248456 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585c98a5-df34-426e-8b84-9f077fb075ce admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:03.357817 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 968/1902] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:26:03 2022] POST /identity/v3/projects => generated 369 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:03.362612 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c98a5-df34-426e-8b84-9f077fb075ce admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:03.399375 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8d49b169-9cb8-4fd5-9f9f-e91d0a295fe9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '429b5bf1eccd442a8aa5f561440de79a'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:03.426756 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8d49b169-9cb8-4fd5-9f9f-e91d0a295fe9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '429b5bf1eccd442a8aa5f561440de79a'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:03.466332 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c98a5-df34-426e-8b84-9f077fb075ce admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:03.472706 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660273f-e23e-4655-97f8-7a88f3fd6d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:03.473379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660273f-e23e-4655-97f8-7a88f3fd6d6f admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:03.473662 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660273f-e23e-4655-97f8-7a88f3fd6d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:03.474033 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660273f-e23e-4655-97f8-7a88f3fd6d6f admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:03.474730 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c660273f-e23e-4655-97f8-7a88f3fd6d6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:03.477040 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c660273f-e23e-4655-97f8-7a88f3fd6d6f admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:03.582406 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8d49b169-9cb8-4fd5-9f9f-e91d0a295fe9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '429b5bf1eccd442a8aa5f561440de79a'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:03.583019 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8d49b169-9cb8-4fd5-9f9f-e91d0a295fe9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '429b5bf1eccd442a8aa5f561440de79a'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:03.591641 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8d49b169-9cb8-4fd5-9f9f-e91d0a295fe9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '429b5bf1eccd442a8aa5f561440de79a'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:03.598685 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 935/1903] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:26:03 2022] DELETE /identity/v3/users/429b5bf1eccd442a8aa5f561440de79a => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:03.625909 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 969/1904] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:26:03 2022] POST /identity/v3/users => generated 285 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:26:03.630648 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d49b169-9cb8-4fd5-9f9f-e91d0a295fe9 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:03.632060 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8d49b169-9cb8-4fd5-9f9f-e91d0a295fe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:03.700842 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d49b169-9cb8-4fd5-9f9f-e91d0a295fe9 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:03.705612 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23259852-618c-4f25-9646-fb2a0e8e48d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:03.706246 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23259852-618c-4f25-9646-fb2a0e8e48d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:03.706542 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23259852-618c-4f25-9646-fb2a0e8e48d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:03.706970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23259852-618c-4f25-9646-fb2a0e8e48d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:03.707675 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23259852-618c-4f25-9646-fb2a0e8e48d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:03.709315 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23259852-618c-4f25-9646-fb2a0e8e48d6 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:03.719836 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 936/1905] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:26:03.724568 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660273f-e23e-4655-97f8-7a88f3fd6d6f admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:03.731218 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:03.735144 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660273f-e23e-4655-97f8-7a88f3fd6d6f admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:03.740630 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9151e9c-3bbf-45b7-8422-525b7dfbf500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:03.741320 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9151e9c-3bbf-45b7-8422-525b7dfbf500 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:03.741659 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9151e9c-3bbf-45b7-8422-525b7dfbf500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:03.742002 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9151e9c-3bbf-45b7-8422-525b7dfbf500 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:03.742763 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9151e9c-3bbf-45b7-8422-525b7dfbf500 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:03.744449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9151e9c-3bbf-45b7-8422-525b7dfbf500 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:03.755589 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 970/1906] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:26:03.759935 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23259852-618c-4f25-9646-fb2a0e8e48d6 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:03.771307 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23259852-618c-4f25-9646-fb2a0e8e48d6 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:03.776723 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ab241-511f-43a8-a249-07ef85f82b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:03.777399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ab241-511f-43a8-a249-07ef85f82b94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:03.777668 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ab241-511f-43a8-a249-07ef85f82b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:03.777936 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ab241-511f-43a8-a249-07ef85f82b94 admin admin] PATH_INFO: `/v3/projects/0a585aa9a489497a9f27cfbfbfa59f16/users/ea9b4eb012d844af985749a5b2f1e82f/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:03.801280 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ab241-511f-43a8-a249-07ef85f82b94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a585aa9a489497a9f27cfbfbfa59f16, user_id=ea9b4eb012d844af985749a5b2f1e82f, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:03.804362 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ab241-511f-43a8-a249-07ef85f82b94 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:03.917723 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 937/1907] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:26:03 2022] PUT /identity/v3/projects/0a585aa9a489497a9f27cfbfbfa59f16/users/ea9b4eb012d844af985749a5b2f1e82f/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:03.930036 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c8bb5-3156-4870-8d07-b1bbc2b91da0 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:03.930890 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c8bb5-3156-4870-8d07-b1bbc2b91da0 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:03.931545 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c8bb5-3156-4870-8d07-b1bbc2b91da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:03.985072 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-354c8bb5-3156-4870-8d07-b1bbc2b91da0 None None] MFA Rules not processed for user `ea9b4eb012d844af985749a5b2f1e82f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:03.986987 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-354c8bb5-3156-4870-8d07-b1bbc2b91da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:04.085040 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 971/1908] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:04.086296 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ab241-511f-43a8-a249-07ef85f82b94 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:04.167760 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ab241-511f-43a8-a249-07ef85f82b94 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:04.175153 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6babcb5-c9ba-466c-b12e-1cefe4f7d366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:04.177616 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6babcb5-c9ba-466c-b12e-1cefe4f7d366 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:04.177892 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6babcb5-c9ba-466c-b12e-1cefe4f7d366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:04.178179 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6babcb5-c9ba-466c-b12e-1cefe4f7d366 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:04.179206 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6babcb5-c9ba-466c-b12e-1cefe4f7d366 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:04.181044 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6babcb5-c9ba-466c-b12e-1cefe4f7d366 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:04.257804 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c8bb5-3156-4870-8d07-b1bbc2b91da0 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:04.259594 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-354c8bb5-3156-4870-8d07-b1bbc2b91da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:04.271381 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 938/1909] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:26:04 2022] POST /identity/v3/projects => generated 369 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:04.275749 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6babcb5-c9ba-466c-b12e-1cefe4f7d366 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:04.280904 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c8bb5-3156-4870-8d07-b1bbc2b91da0 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:04.351353 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71252940-9fb7-43e8-9535-eeca9d760d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:04.352060 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71252940-9fb7-43e8-9535-eeca9d760d0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:04.352342 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71252940-9fb7-43e8-9535-eeca9d760d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:04.352507 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6babcb5-c9ba-466c-b12e-1cefe4f7d366 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:04.352645 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71252940-9fb7-43e8-9535-eeca9d760d0a admin admin] PATH_INFO: `/v3/projects/53c0ecad63494c169c6cce1fcbdac30c` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:04.353868 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71252940-9fb7-43e8-9535-eeca9d760d0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=53c0ecad63494c169c6cce1fcbdac30c)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:04.356151 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71252940-9fb7-43e8-9535-eeca9d760d0a admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:04.369735 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b21e2b-0d98-4911-b04f-315e6d028152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:04.369735 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b21e2b-0d98-4911-b04f-315e6d028152 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:04.369735 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b21e2b-0d98-4911-b04f-315e6d028152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:04.369735 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b21e2b-0d98-4911-b04f-315e6d028152 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:04.370139 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b21e2b-0d98-4911-b04f-315e6d028152 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:04.370139 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b21e2b-0d98-4911-b04f-315e6d028152 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:04.382905 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:04.423122 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 939/1910] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:26:04 2022] POST /identity/v3/users => generated 291 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:26:04.427806 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b21e2b-0d98-4911-b04f-315e6d028152 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:04.438647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b21e2b-0d98-4911-b04f-315e6d028152 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:04.454152 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-71252940-9fb7-43e8-9535-eeca9d760d0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53c0ecad63494c169c6cce1fcbdac30c', 'request_id': 'req-71252940-9fb7-43e8-9535-eeca9d760d0a'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:04.461472 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-71252940-9fb7-43e8-9535-eeca9d760d0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53c0ecad63494c169c6cce1fcbdac30c', 'request_id': 'req-71252940-9fb7-43e8-9535-eeca9d760d0a'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:04.480188 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-71252940-9fb7-43e8-9535-eeca9d760d0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53c0ecad63494c169c6cce1fcbdac30c', 'request_id': 'req-71252940-9fb7-43e8-9535-eeca9d760d0a'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:04.497521 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa2026f-1bfd-40af-9759-2223403d95ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:04.498158 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2026f-1bfd-40af-9759-2223403d95ee admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:04.498435 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2026f-1bfd-40af-9759-2223403d95ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:04.498706 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2026f-1bfd-40af-9759-2223403d95ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:04.499405 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa2026f-1bfd-40af-9759-2223403d95ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:04.501064 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa2026f-1bfd-40af-9759-2223403d95ee admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:04.511343 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 940/1911] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:26:04.520690 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa2026f-1bfd-40af-9759-2223403d95ee admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:04.526036 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa2026f-1bfd-40af-9759-2223403d95ee admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:04.530949 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8734960f-f2b9-489c-ad6c-265aa4baf0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:04.531607 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734960f-f2b9-489c-ad6c-265aa4baf0fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:04.531885 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734960f-f2b9-489c-ad6c-265aa4baf0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:04.532204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734960f-f2b9-489c-ad6c-265aa4baf0fd admin admin] PATH_INFO: `/v3/projects/a24923573225436a99c422f9bdbedabd/users/6c62952d65ef42128eb4dfb01ec1789f/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:04.543191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-71252940-9fb7-43e8-9535-eeca9d760d0a admin admin] The token cache is being invalidate because project 53c0ecad63494c169c6cce1fcbdac30c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:26:04.543570 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-71252940-9fb7-43e8-9535-eeca9d760d0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:04.544337 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-71252940-9fb7-43e8-9535-eeca9d760d0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:04.545979 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 972/1912] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:26:04 2022] DELETE /identity/v3/projects/53c0ecad63494c169c6cce1fcbdac30c => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:26:04.551134 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8734960f-f2b9-489c-ad6c-265aa4baf0fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a24923573225436a99c422f9bdbedabd, user_id=6c62952d65ef42128eb4dfb01ec1789f, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:04.551314 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7c405-c334-43d6-a43f-16f170377f99 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:04.551550 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7c405-c334-43d6-a43f-16f170377f99 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:04.551834 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7c405-c334-43d6-a43f-16f170377f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:04.554118 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8734960f-f2b9-489c-ad6c-265aa4baf0fd admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:04.572746 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-03a7c405-c334-43d6-a43f-16f170377f99 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:04.573966 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-03a7c405-c334-43d6-a43f-16f170377f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:04.582590 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 973/1913] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:04.590025 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f24cf-0258-4f13-ba40-a4d045b3e8e9 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:04.590311 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f24cf-0258-4f13-ba40-a4d045b3e8e9 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:04.590590 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f24cf-0258-4f13-ba40-a4d045b3e8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:04.610422 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-1a2f24cf-0258-4f13-ba40-a4d045b3e8e9 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:04.611630 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-1a2f24cf-0258-4f13-ba40-a4d045b3e8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:04.616702 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 941/1914] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:26:04 2022] PUT /identity/v3/projects/a24923573225436a99c422f9bdbedabd/users/6c62952d65ef42128eb4dfb01ec1789f/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:26:04.625446 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:04.630428 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8734960f-f2b9-489c-ad6c-265aa4baf0fd admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:04.632107 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8734960f-f2b9-489c-ad6c-265aa4baf0fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:04.689731 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 974/1915] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:26:04.693606 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2f24cf-0258-4f13-ba40-a4d045b3e8e9 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:04.703912 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8734960f-f2b9-489c-ad6c-265aa4baf0fd admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:04.706634 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2f24cf-0258-4f13-ba40-a4d045b3e8e9 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:04.709410 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef58a58-24e6-426d-8648-8d9fcdc67d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:04.710054 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef58a58-24e6-426d-8648-8d9fcdc67d32 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:04.710333 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef58a58-24e6-426d-8648-8d9fcdc67d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:04.710608 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef58a58-24e6-426d-8648-8d9fcdc67d32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:04.711509 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef58a58-24e6-426d-8648-8d9fcdc67d32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:04.712834 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d596eb-5f49-4c14-99f9-e7dbdf17f3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:04.712969 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef58a58-24e6-426d-8648-8d9fcdc67d32 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:04.713633 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d596eb-5f49-4c14-99f9-e7dbdf17f3b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:04.714038 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d596eb-5f49-4c14-99f9-e7dbdf17f3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:04.714404 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d596eb-5f49-4c14-99f9-e7dbdf17f3b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:04.715314 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d596eb-5f49-4c14-99f9-e7dbdf17f3b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:26:04.715706 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d596eb-5f49-4c14-99f9-e7dbdf17f3b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:04.717661 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d596eb-5f49-4c14-99f9-e7dbdf17f3b4 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:04.725261 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 942/1916] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:26:04.729124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef58a58-24e6-426d-8648-8d9fcdc67d32 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:04.738861 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 975/1917] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:26:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:26:04.742754 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d596eb-5f49-4c14-99f9-e7dbdf17f3b4 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:04.745125 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef58a58-24e6-426d-8648-8d9fcdc67d32 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:04.750643 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b515c-03ff-4efc-9e15-23dc919f928b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:04.751849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b515c-03ff-4efc-9e15-23dc919f928b admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:04.751849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b515c-03ff-4efc-9e15-23dc919f928b admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:04.752042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b515c-03ff-4efc-9e15-23dc919f928b admin admin] PATH_INFO: `/v3/projects/a24923573225436a99c422f9bdbedabd/users/6c62952d65ef42128eb4dfb01ec1789f/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:04.755032 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39b515c-03ff-4efc-9e15-23dc919f928b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a24923573225436a99c422f9bdbedabd, user_id=6c62952d65ef42128eb4dfb01ec1789f, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:04.757901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39b515c-03ff-4efc-9e15-23dc919f928b admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:04.759557 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d596eb-5f49-4c14-99f9-e7dbdf17f3b4 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:04.765877 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af782a-3564-44e6-90d3-25cb85f04322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:04.766073 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af782a-3564-44e6-90d3-25cb85f04322 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:04.766296 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af782a-3564-44e6-90d3-25cb85f04322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:04.766505 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af782a-3564-44e6-90d3-25cb85f04322 admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:04.767181 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af782a-3564-44e6-90d3-25cb85f04322 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:04.769924 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af782a-3564-44e6-90d3-25cb85f04322 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:04.774898 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 943/1918] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:26:04 2022] PUT /identity/v3/projects/a24923573225436a99c422f9bdbedabd/users/6c62952d65ef42128eb4dfb01ec1789f/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:26:04.784516 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa1c17-b2af-47b5-b620-5989d232c1e1 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:04.784829 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa1c17-b2af-47b5-b620-5989d232c1e1 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:04.785153 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa1c17-b2af-47b5-b620-5989d232c1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:04.829744 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-d7fa1c17-b2af-47b5-b620-5989d232c1e1 None None] MFA Rules not processed for user `6c62952d65ef42128eb4dfb01ec1789f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:04.831082 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d7fa1c17-b2af-47b5-b620-5989d232c1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:04.895269 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 976/1919] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:26:04 2022] POST /identity/v3/projects => generated 387 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:04.896823 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af782a-3564-44e6-90d3-25cb85f04322 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:04.936466 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 944/1920] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:04 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:04.950611 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa1c17-b2af-47b5-b620-5989d232c1e1 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:04.950611 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d7fa1c17-b2af-47b5-b620-5989d232c1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:04.983036 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af782a-3564-44e6-90d3-25cb85f04322 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:04.987987 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2063e44e-bedd-46d0-ab39-97e705f61cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:04.988715 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2063e44e-bedd-46d0-ab39-97e705f61cf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:04.989068 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2063e44e-bedd-46d0-ab39-97e705f61cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:04.989416 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2063e44e-bedd-46d0-ab39-97e705f61cf7 admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:04.990174 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2063e44e-bedd-46d0-ab39-97e705f61cf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:04.992603 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2063e44e-bedd-46d0-ab39-97e705f61cf7 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:05.011906 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa1c17-b2af-47b5-b620-5989d232c1e1 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:05.066683 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e163ad6c-5e5f-4b8c-b506-77436ed95ab6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:05.067305 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163ad6c-5e5f-4b8c-b506-77436ed95ab6 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:05.067579 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163ad6c-5e5f-4b8c-b506-77436ed95ab6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:05.067847 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163ad6c-5e5f-4b8c-b506-77436ed95ab6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:05.117637 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 977/1921] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:26:04 2022] POST /identity/v3/users => generated 294 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:26:05.119020 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2063e44e-bedd-46d0-ab39-97e705f61cf7 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:05.128130 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e163ad6c-5e5f-4b8c-b506-77436ed95ab6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:05.129875 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2063e44e-bedd-46d0-ab39-97e705f61cf7 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:05.130338 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e163ad6c-5e5f-4b8c-b506-77436ed95ab6 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:05.134517 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158fec18-c5fb-43fc-8505-d90cfce53940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:05.135112 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158fec18-c5fb-43fc-8505-d90cfce53940 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:05.135411 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158fec18-c5fb-43fc-8505-d90cfce53940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:05.135680 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158fec18-c5fb-43fc-8505-d90cfce53940 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:05.136360 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158fec18-c5fb-43fc-8505-d90cfce53940 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:05.137954 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158fec18-c5fb-43fc-8505-d90cfce53940 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:05.140247 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 945/1922] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:04 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 14:26:05.150795 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 978/1923] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:26:05.155091 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e163ad6c-5e5f-4b8c-b506-77436ed95ab6 service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:05.165895 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e163ad6c-5e5f-4b8c-b506-77436ed95ab6 service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:05.170868 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad49867-f390-476e-bdda-d4040c4bc53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:05.171502 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad49867-f390-476e-bdda-d4040c4bc53e admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:05.175594 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad49867-f390-476e-bdda-d4040c4bc53e admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:05.175594 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad49867-f390-476e-bdda-d4040c4bc53e admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:05.175594 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad49867-f390-476e-bdda-d4040c4bc53e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:05.175594 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad49867-f390-476e-bdda-d4040c4bc53e admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:05.184620 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 946/1924] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:26:05.188974 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158fec18-c5fb-43fc-8505-d90cfce53940 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:05.199395 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158fec18-c5fb-43fc-8505-d90cfce53940 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:05.204273 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747aa879-7266-4010-97b5-3e3a8d1b20cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:05.204729 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747aa879-7266-4010-97b5-3e3a8d1b20cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:05.205010 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747aa879-7266-4010-97b5-3e3a8d1b20cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:05.205289 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747aa879-7266-4010-97b5-3e3a8d1b20cc admin admin] PATH_INFO: `/v3/projects/8544a4e51cf44b0f96c26f0caae1f026/users/2ac0f0922f2f4b2189075c17574eff19/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:05.220821 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747aa879-7266-4010-97b5-3e3a8d1b20cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8544a4e51cf44b0f96c26f0caae1f026, user_id=2ac0f0922f2f4b2189075c17574eff19, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:05.223598 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747aa879-7266-4010-97b5-3e3a8d1b20cc admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:05.349609 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 979/1925] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:26:05 2022] PUT /identity/v3/projects/8544a4e51cf44b0f96c26f0caae1f026/users/2ac0f0922f2f4b2189075c17574eff19/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:05.360222 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4918d7e2-72f1-4939-9453-09ec98e49b4f None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:05.360559 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4918d7e2-72f1-4939-9453-09ec98e49b4f None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:05.360839 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4918d7e2-72f1-4939-9453-09ec98e49b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:05.393201 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-4918d7e2-72f1-4939-9453-09ec98e49b4f None None] MFA Rules not processed for user `2ac0f0922f2f4b2189075c17574eff19`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:05.394671 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-4918d7e2-72f1-4939-9453-09ec98e49b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:05.471412 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 947/1926] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:26:05.475981 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747aa879-7266-4010-97b5-3e3a8d1b20cc admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:05.545228 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747aa879-7266-4010-97b5-3e3a8d1b20cc admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:05.549974 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0622da90-0b4e-417a-a66b-5070bd9038e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:05.551176 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622da90-0b4e-417a-a66b-5070bd9038e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:05.551446 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622da90-0b4e-417a-a66b-5070bd9038e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:05.551713 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622da90-0b4e-417a-a66b-5070bd9038e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:05.552447 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0622da90-0b4e-417a-a66b-5070bd9038e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:05.554367 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0622da90-0b4e-417a-a66b-5070bd9038e3 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:05.561065 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:05.662626 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 980/1927] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:26:05 2022] POST /identity/v3/projects => generated 385 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:05.663812 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4918d7e2-72f1-4939-9453-09ec98e49b4f None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:05.674818 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4918d7e2-72f1-4939-9453-09ec98e49b4f None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:05.745669 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c5b302-76d8-4dbe-b2c7-16c513b82f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:05.746288 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5b302-76d8-4dbe-b2c7-16c513b82f2c admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:05.746562 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5b302-76d8-4dbe-b2c7-16c513b82f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:05.746885 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5b302-76d8-4dbe-b2c7-16c513b82f2c admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:05.747486 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c5b302-76d8-4dbe-b2c7-16c513b82f2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:05.749397 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c5b302-76d8-4dbe-b2c7-16c513b82f2c admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:05.755717 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:05.846271 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 948/1928] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:26:05 2022] POST /identity/v3/users => generated 299 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:05.850405 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0622da90-0b4e-417a-a66b-5070bd9038e3 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:05.860931 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0622da90-0b4e-417a-a66b-5070bd9038e3 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:05.865676 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff30b78-05bd-4c1a-8d5a-ed947333544a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:05.866282 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff30b78-05bd-4c1a-8d5a-ed947333544a admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:05.866559 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff30b78-05bd-4c1a-8d5a-ed947333544a admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:05.866832 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff30b78-05bd-4c1a-8d5a-ed947333544a admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:05.867580 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff30b78-05bd-4c1a-8d5a-ed947333544a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:05.869199 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff30b78-05bd-4c1a-8d5a-ed947333544a admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:05.880029 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 981/1929] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:26:05.884300 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c5b302-76d8-4dbe-b2c7-16c513b82f2c admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:05.896440 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c5b302-76d8-4dbe-b2c7-16c513b82f2c admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:05.901106 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef665156-edb8-4416-8c32-80fa1edf91d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:05.901816 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef665156-edb8-4416-8c32-80fa1edf91d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:05.903604 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef665156-edb8-4416-8c32-80fa1edf91d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:05.903604 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef665156-edb8-4416-8c32-80fa1edf91d7 admin admin] PATH_INFO: `/v3/projects/94491ca872474b768ba34c60171be988/users/8ddab65911fa427da1da60267952c4c1/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:05.918641 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef665156-edb8-4416-8c32-80fa1edf91d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94491ca872474b768ba34c60171be988, user_id=8ddab65911fa427da1da60267952c4c1, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:05.921454 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef665156-edb8-4416-8c32-80fa1edf91d7 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:06.010052 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 949/1930] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:26:05 2022] PUT /identity/v3/projects/94491ca872474b768ba34c60171be988/users/8ddab65911fa427da1da60267952c4c1/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:06.010202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff30b78-05bd-4c1a-8d5a-ed947333544a admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:06.018066 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff30b78-05bd-4c1a-8d5a-ed947333544a admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:06.094904 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5895b-0b4b-4ad5-886b-d4aaeeb0ae0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:06.095526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5895b-0b4b-4ad5-886b-d4aaeeb0ae0b admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:06.095839 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5895b-0b4b-4ad5-886b-d4aaeeb0ae0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:06.096224 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5895b-0b4b-4ad5-886b-d4aaeeb0ae0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:06.096947 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f5895b-0b4b-4ad5-886b-d4aaeeb0ae0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:06.098970 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f5895b-0b4b-4ad5-886b-d4aaeeb0ae0b admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:06.112057 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 982/1931] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:26:06.116528 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef665156-edb8-4416-8c32-80fa1edf91d7 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:06.128000 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef665156-edb8-4416-8c32-80fa1edf91d7 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:06.137851 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:06.138160 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d756445-acfd-4480-a567-4aff3cf97344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:06.138160 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d756445-acfd-4480-a567-4aff3cf97344 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:06.138160 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d756445-acfd-4480-a567-4aff3cf97344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:06.138160 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d756445-acfd-4480-a567-4aff3cf97344 admin admin] PATH_INFO: `/v3/projects/94491ca872474b768ba34c60171be988/users/8ddab65911fa427da1da60267952c4c1/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:06.138480 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d756445-acfd-4480-a567-4aff3cf97344 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94491ca872474b768ba34c60171be988, user_id=8ddab65911fa427da1da60267952c4c1, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:06.139938 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d756445-acfd-4480-a567-4aff3cf97344 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:06.150981 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 950/1932] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:26:06 2022] PUT /identity/v3/projects/94491ca872474b768ba34c60171be988/users/8ddab65911fa427da1da60267952c4c1/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:06.162261 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d677c-0f6a-41ea-9ffd-e14251051d8c None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:06.162547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d677c-0f6a-41ea-9ffd-e14251051d8c None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:06.162823 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d677c-0f6a-41ea-9ffd-e14251051d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:06.193870 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-4c5d677c-0f6a-41ea-9ffd-e14251051d8c None None] MFA Rules not processed for user `8ddab65911fa427da1da60267952c4c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:06.195161 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-4c5d677c-0f6a-41ea-9ffd-e14251051d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:06.229820 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:06.267703 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 983/1933] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:06 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:06.301161 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d756445-acfd-4480-a567-4aff3cf97344 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:06.363528 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d756445-acfd-4480-a567-4aff3cf97344 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:06.363528 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a48106-671f-4169-9dba-be6d1d928001 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:06.363528 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a48106-671f-4169-9dba-be6d1d928001 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:06.363927 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a48106-671f-4169-9dba-be6d1d928001 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:06.364195 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a48106-671f-4169-9dba-be6d1d928001 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:06.374326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a48106-671f-4169-9dba-be6d1d928001 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:06.375760 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a48106-671f-4169-9dba-be6d1d928001 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:06.386427 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 951/1934] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:06 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:26:06.582396 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:06.811942 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:07.410875 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:07.557089 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:07.656090 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5d677c-0f6a-41ea-9ffd-e14251051d8c None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:07.722273 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5d677c-0f6a-41ea-9ffd-e14251051d8c None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:07.729981 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4d293-3942-4613-895d-be66750b8cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:07.730653 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4d293-3942-4613-895d-be66750b8cab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:07.730896 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4d293-3942-4613-895d-be66750b8cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:07.731166 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4d293-3942-4613-895d-be66750b8cab admin admin] PATH_INFO: `/v3/users/2ac0f0922f2f4b2189075c17574eff19` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:07.732352 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d4d293-3942-4613-895d-be66750b8cab admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ac0f0922f2f4b2189075c17574eff19)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:07.734325 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d4d293-3942-4613-895d-be66750b8cab admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:07.766660 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:07.955846 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:07.958674 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-04d4d293-3942-4613-895d-be66750b8cab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ac0f0922f2f4b2189075c17574eff19'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:07.965883 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-04d4d293-3942-4613-895d-be66750b8cab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ac0f0922f2f4b2189075c17574eff19'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:07.966926 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-04d4d293-3942-4613-895d-be66750b8cab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ac0f0922f2f4b2189075c17574eff19'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:07.967476 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-04d4d293-3942-4613-895d-be66750b8cab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ac0f0922f2f4b2189075c17574eff19'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:07.980854 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-04d4d293-3942-4613-895d-be66750b8cab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ac0f0922f2f4b2189075c17574eff19'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:08.069342 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:08.075608 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 984/1935] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:26:07 2022] DELETE /identity/v3/users/2ac0f0922f2f4b2189075c17574eff19 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:08.080243 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ff9e3-9629-4677-8b76-6b474577c531 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:08.080577 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ff9e3-9629-4677-8b76-6b474577c531 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:08.080860 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ff9e3-9629-4677-8b76-6b474577c531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:08.101560 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-616ff9e3-9629-4677-8b76-6b474577c531 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:08.102840 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-616ff9e3-9629-4677-8b76-6b474577c531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:08.162676 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 952/1936] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:08.173542 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4d293-3942-4613-895d-be66750b8cab admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:08.173903 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-04d4d293-3942-4613-895d-be66750b8cab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:08.198582 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ff9e3-9629-4677-8b76-6b474577c531 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:08.199967 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-616ff9e3-9629-4677-8b76-6b474577c531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:08.212259 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ff9e3-9629-4677-8b76-6b474577c531 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:08.217867 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce82145e-cf53-479b-91c0-ce1f028381c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:08.218512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce82145e-cf53-479b-91c0-ce1f028381c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:08.218789 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce82145e-cf53-479b-91c0-ce1f028381c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:08.219057 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce82145e-cf53-479b-91c0-ce1f028381c4 admin admin] PATH_INFO: `/v3/users/ea9b4eb012d844af985749a5b2f1e82f` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:08.220232 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce82145e-cf53-479b-91c0-ce1f028381c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea9b4eb012d844af985749a5b2f1e82f)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:08.222715 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce82145e-cf53-479b-91c0-ce1f028381c4 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:08.234567 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4d293-3942-4613-895d-be66750b8cab admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:08.238730 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60db8774-3e98-4063-b26d-ba796df52ff4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:08.239346 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60db8774-3e98-4063-b26d-ba796df52ff4 service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:08.239630 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60db8774-3e98-4063-b26d-ba796df52ff4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:08.239902 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60db8774-3e98-4063-b26d-ba796df52ff4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:08.249008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60db8774-3e98-4063-b26d-ba796df52ff4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:08.250448 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60db8774-3e98-4063-b26d-ba796df52ff4 service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:08.260093 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 985/1937] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:26:08.406288 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ce82145e-cf53-479b-91c0-ce1f028381c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea9b4eb012d844af985749a5b2f1e82f'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:08.418648 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ce82145e-cf53-479b-91c0-ce1f028381c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea9b4eb012d844af985749a5b2f1e82f'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:08.501608 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ce82145e-cf53-479b-91c0-ce1f028381c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea9b4eb012d844af985749a5b2f1e82f'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:08.502402 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ce82145e-cf53-479b-91c0-ce1f028381c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea9b4eb012d844af985749a5b2f1e82f'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:08.509969 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ce82145e-cf53-479b-91c0-ce1f028381c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea9b4eb012d844af985749a5b2f1e82f'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:08.520637 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 953/1938] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:26:08 2022] DELETE /identity/v3/users/ea9b4eb012d844af985749a5b2f1e82f => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:26:08.526644 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e98cb2-b049-4f6d-9d97-105b80852b42 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:08.527103 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e98cb2-b049-4f6d-9d97-105b80852b42 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:08.527248 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e98cb2-b049-4f6d-9d97-105b80852b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:08.547660 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-37e98cb2-b049-4f6d-9d97-105b80852b42 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:08.548932 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-37e98cb2-b049-4f6d-9d97-105b80852b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:08.611400 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 986/1939] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:08.621583 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce82145e-cf53-479b-91c0-ce1f028381c4 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:08.622956 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ce82145e-cf53-479b-91c0-ce1f028381c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:08.680465 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce82145e-cf53-479b-91c0-ce1f028381c4 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:08.684095 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc4af49-b70b-421e-a874-a4f7897cf2a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:08.684753 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4af49-b70b-421e-a874-a4f7897cf2a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:08.685025 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4af49-b70b-421e-a874-a4f7897cf2a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:08.685291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4af49-b70b-421e-a874-a4f7897cf2a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:08.695754 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc4af49-b70b-421e-a874-a4f7897cf2a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:08.697208 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc4af49-b70b-421e-a874-a4f7897cf2a5 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:08.708314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 954/1940] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:26:08.721773 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e98cb2-b049-4f6d-9d97-105b80852b42 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:08.724130 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-37e98cb2-b049-4f6d-9d97-105b80852b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:08.738666 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e98cb2-b049-4f6d-9d97-105b80852b42 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:08.743602 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e910aab-311e-4eb2-9a37-a5f681893a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:08.744441 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e910aab-311e-4eb2-9a37-a5f681893a8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:08.744873 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e910aab-311e-4eb2-9a37-a5f681893a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:08.745149 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e910aab-311e-4eb2-9a37-a5f681893a8d admin admin] PATH_INFO: `/v3/projects/8544a4e51cf44b0f96c26f0caae1f026` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:08.746329 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e910aab-311e-4eb2-9a37-a5f681893a8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8544a4e51cf44b0f96c26f0caae1f026)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:08.748388 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e910aab-311e-4eb2-9a37-a5f681893a8d admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:08.783341 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:08.827496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:08.845934 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-6e910aab-311e-4eb2-9a37-a5f681893a8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8544a4e51cf44b0f96c26f0caae1f026', 'request_id': 'req-6e910aab-311e-4eb2-9a37-a5f681893a8d'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:08.852135 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-6e910aab-311e-4eb2-9a37-a5f681893a8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8544a4e51cf44b0f96c26f0caae1f026', 'request_id': 'req-6e910aab-311e-4eb2-9a37-a5f681893a8d'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:08.858145 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-6e910aab-311e-4eb2-9a37-a5f681893a8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8544a4e51cf44b0f96c26f0caae1f026', 'request_id': 'req-6e910aab-311e-4eb2-9a37-a5f681893a8d'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:08.926913 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-6e910aab-311e-4eb2-9a37-a5f681893a8d admin admin] The token cache is being invalidate because project 8544a4e51cf44b0f96c26f0caae1f026 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:26:08.927268 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-6e910aab-311e-4eb2-9a37-a5f681893a8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:08.929912 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-6e910aab-311e-4eb2-9a37-a5f681893a8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:08.929912 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 987/1941] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:26:08 2022] DELETE /identity/v3/projects/8544a4e51cf44b0f96c26f0caae1f026 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:08.933784 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc4af49-b70b-421e-a874-a4f7897cf2a5 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:08.947708 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc4af49-b70b-421e-a874-a4f7897cf2a5 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:09.011874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8be228-64a1-43a1-9f0c-19a11ca54b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:09.012596 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8be228-64a1-43a1-9f0c-19a11ca54b3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:09.012874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8be228-64a1-43a1-9f0c-19a11ca54b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:09.013169 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8be228-64a1-43a1-9f0c-19a11ca54b3d admin admin] PATH_INFO: `/v3/users/8ddab65911fa427da1da60267952c4c1` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:09.014337 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8be228-64a1-43a1-9f0c-19a11ca54b3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ddab65911fa427da1da60267952c4c1)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:09.016370 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8be228-64a1-43a1-9f0c-19a11ca54b3d admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:09.172321 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ca8be228-64a1-43a1-9f0c-19a11ca54b3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ddab65911fa427da1da60267952c4c1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:09.188397 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ca8be228-64a1-43a1-9f0c-19a11ca54b3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ddab65911fa427da1da60267952c4c1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:09.260902 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ca8be228-64a1-43a1-9f0c-19a11ca54b3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ddab65911fa427da1da60267952c4c1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:09.261803 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ca8be228-64a1-43a1-9f0c-19a11ca54b3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ddab65911fa427da1da60267952c4c1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:09.270766 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ca8be228-64a1-43a1-9f0c-19a11ca54b3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ddab65911fa427da1da60267952c4c1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:09.277739 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 955/1942] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:26:08 2022] DELETE /identity/v3/users/8ddab65911fa427da1da60267952c4c1 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:09.280302 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:09.333315 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:09.542699 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e910aab-311e-4eb2-9a37-a5f681893a8d admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:09.544623 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-6e910aab-311e-4eb2-9a37-a5f681893a8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:09.582267 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e910aab-311e-4eb2-9a37-a5f681893a8d admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:09.654842 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70c992-b797-4d91-b403-b289ae13650c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:09.656503 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70c992-b797-4d91-b403-b289ae13650c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:09.656895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70c992-b797-4d91-b403-b289ae13650c admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:09.657351 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70c992-b797-4d91-b403-b289ae13650c admin admin] PATH_INFO: `/v3/projects/0a585aa9a489497a9f27cfbfbfa59f16` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:09.658659 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc70c992-b797-4d91-b403-b289ae13650c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a585aa9a489497a9f27cfbfbfa59f16)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:09.661936 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc70c992-b797-4d91-b403-b289ae13650c admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:09.812679 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-bc70c992-b797-4d91-b403-b289ae13650c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a585aa9a489497a9f27cfbfbfa59f16', 'request_id': 'req-bc70c992-b797-4d91-b403-b289ae13650c'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:09.818687 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-bc70c992-b797-4d91-b403-b289ae13650c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a585aa9a489497a9f27cfbfbfa59f16', 'request_id': 'req-bc70c992-b797-4d91-b403-b289ae13650c'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:09.824076 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-bc70c992-b797-4d91-b403-b289ae13650c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a585aa9a489497a9f27cfbfbfa59f16', 'request_id': 'req-bc70c992-b797-4d91-b403-b289ae13650c'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:09.864787 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8be228-64a1-43a1-9f0c-19a11ca54b3d admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:09.866507 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ca8be228-64a1-43a1-9f0c-19a11ca54b3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:09.881698 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8be228-64a1-43a1-9f0c-19a11ca54b3d admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:09.908003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-bc70c992-b797-4d91-b403-b289ae13650c admin admin] The token cache is being invalidate because project 0a585aa9a489497a9f27cfbfbfa59f16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:26:09.908581 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-bc70c992-b797-4d91-b403-b289ae13650c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:09.910165 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-bc70c992-b797-4d91-b403-b289ae13650c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:09.912015 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 988/1943] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:26:09 2022] DELETE /identity/v3/projects/0a585aa9a489497a9f27cfbfbfa59f16 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:09.916090 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70c992-b797-4d91-b403-b289ae13650c admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:09.917556 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-bc70c992-b797-4d91-b403-b289ae13650c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:09.958414 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1926b67c-e223-4177-8957-3496ffbc1ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:09.959063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926b67c-e223-4177-8957-3496ffbc1ccd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:09.960797 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926b67c-e223-4177-8957-3496ffbc1ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:09.961084 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926b67c-e223-4177-8957-3496ffbc1ccd admin admin] PATH_INFO: `/v3/projects/94491ca872474b768ba34c60171be988` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:09.965196 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1926b67c-e223-4177-8957-3496ffbc1ccd admin admin] RBAC: Authorizing `identity:delete_project(project_id=94491ca872474b768ba34c60171be988)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:09.968311 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1926b67c-e223-4177-8957-3496ffbc1ccd admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:09.989987 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70c992-b797-4d91-b403-b289ae13650c admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:09.994903 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e364270-0f01-4601-871b-af481275ad4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:09.995556 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e364270-0f01-4601-871b-af481275ad4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:09.995838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e364270-0f01-4601-871b-af481275ad4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:09.996112 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e364270-0f01-4601-871b-af481275ad4c admin admin] PATH_INFO: `/v3/users/6c62952d65ef42128eb4dfb01ec1789f` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:09.997367 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e364270-0f01-4601-871b-af481275ad4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c62952d65ef42128eb4dfb01ec1789f)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:09.999315 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e364270-0f01-4601-871b-af481275ad4c admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:10.050005 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:10.105384 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1926b67c-e223-4177-8957-3496ffbc1ccd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94491ca872474b768ba34c60171be988', 'request_id': 'req-1926b67c-e223-4177-8957-3496ffbc1ccd'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:10.112509 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1926b67c-e223-4177-8957-3496ffbc1ccd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94491ca872474b768ba34c60171be988', 'request_id': 'req-1926b67c-e223-4177-8957-3496ffbc1ccd'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:10.194700 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:10.223028 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1926b67c-e223-4177-8957-3496ffbc1ccd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94491ca872474b768ba34c60171be988', 'request_id': 'req-1926b67c-e223-4177-8957-3496ffbc1ccd'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:10.234894 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1926b67c-e223-4177-8957-3496ffbc1ccd admin admin] The token cache is being invalidate because project 94491ca872474b768ba34c60171be988 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:26:10.235183 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1926b67c-e223-4177-8957-3496ffbc1ccd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:10.235852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1926b67c-e223-4177-8957-3496ffbc1ccd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:10.237347 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-4e364270-0f01-4601-871b-af481275ad4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c62952d65ef42128eb4dfb01ec1789f'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:10.237468 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 956/1944] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:26:09 2022] DELETE /identity/v3/projects/94491ca872474b768ba34c60171be988 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:26:10.242969 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca1b51-d283-4b17-81d2-376e781e571c None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:10.243249 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca1b51-d283-4b17-81d2-376e781e571c None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:10.243524 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca1b51-d283-4b17-81d2-376e781e571c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:10.245570 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-4e364270-0f01-4601-871b-af481275ad4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c62952d65ef42128eb4dfb01ec1789f'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:10.248853 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-4e364270-0f01-4601-871b-af481275ad4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c62952d65ef42128eb4dfb01ec1789f'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:10.248853 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-4e364270-0f01-4601-871b-af481275ad4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c62952d65ef42128eb4dfb01ec1789f'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:10.264971 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-4e364270-0f01-4601-871b-af481275ad4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c62952d65ef42128eb4dfb01ec1789f'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:10.265130 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-72ca1b51-d283-4b17-81d2-376e781e571c None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:10.266369 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-72ca1b51-d283-4b17-81d2-376e781e571c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:10.328351 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 957/1945] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:10.336026 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9af450-f735-492a-b04e-7e5bf6dd5f20 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:10.336364 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9af450-f735-492a-b04e-7e5bf6dd5f20 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:10.336670 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9af450-f735-492a-b04e-7e5bf6dd5f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:10.358777 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-ed9af450-f735-492a-b04e-7e5bf6dd5f20 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:10.359999 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ed9af450-f735-492a-b04e-7e5bf6dd5f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:10.363712 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 989/1946] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:26:09 2022] DELETE /identity/v3/users/6c62952d65ef42128eb4dfb01ec1789f => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:10.421919 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 958/1947] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:10.426792 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e364270-0f01-4601-871b-af481275ad4c admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:10.437588 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e364270-0f01-4601-871b-af481275ad4c admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:10.450064 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a50ebe-529b-48cc-a993-a98decbf1345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:10.450064 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a50ebe-529b-48cc-a993-a98decbf1345 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:10.450064 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a50ebe-529b-48cc-a993-a98decbf1345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:10.450064 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a50ebe-529b-48cc-a993-a98decbf1345 admin admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:10.451059 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a50ebe-529b-48cc-a993-a98decbf1345 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:26:10.451059 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a50ebe-529b-48cc-a993-a98decbf1345 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:10.451059 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a50ebe-529b-48cc-a993-a98decbf1345 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:10.459713 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 990/1948] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:26:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:26:10.464793 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9af450-f735-492a-b04e-7e5bf6dd5f20 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:10.478687 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9af450-f735-492a-b04e-7e5bf6dd5f20 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:10.486611 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7181f99-c311-4bec-a9b1-ad92e6cac92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:10.488488 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7181f99-c311-4bec-a9b1-ad92e6cac92b admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:10.488774 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7181f99-c311-4bec-a9b1-ad92e6cac92b admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:10.489106 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7181f99-c311-4bec-a9b1-ad92e6cac92b admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:10.489760 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7181f99-c311-4bec-a9b1-ad92e6cac92b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:10.491674 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7181f99-c311-4bec-a9b1-ad92e6cac92b admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:10.573940 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a50ebe-529b-48cc-a993-a98decbf1345 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:10.576007 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-26a50ebe-529b-48cc-a993-a98decbf1345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:10.591088 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a50ebe-529b-48cc-a993-a98decbf1345 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:10.604636 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2b3885-ac6a-4550-996e-e1f9a5d0e7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:10.605310 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b3885-ac6a-4550-996e-e1f9a5d0e7e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:10.606431 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b3885-ac6a-4550-996e-e1f9a5d0e7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:10.608234 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b3885-ac6a-4550-996e-e1f9a5d0e7e1 admin admin] PATH_INFO: `/v3/users/4cfe2d28f8e94d63a334ec53c100e889` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:10.618855 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2b3885-ac6a-4550-996e-e1f9a5d0e7e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cfe2d28f8e94d63a334ec53c100e889)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:10.623319 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 959/1949] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:26:10 2022] POST /identity/v3/projects => generated 381 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:10.624685 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2b3885-ac6a-4550-996e-e1f9a5d0e7e1 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:10.628277 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7181f99-c311-4bec-a9b1-ad92e6cac92b admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:10.732784 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7181f99-c311-4bec-a9b1-ad92e6cac92b admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:10.737783 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f3104-1720-4b6c-b63e-8e4730cf2886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:10.738413 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f3104-1720-4b6c-b63e-8e4730cf2886 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:10.738789 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f3104-1720-4b6c-b63e-8e4730cf2886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:10.738953 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f3104-1720-4b6c-b63e-8e4730cf2886 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:10.739691 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f3104-1720-4b6c-b63e-8e4730cf2886 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:10.741801 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f3104-1720-4b6c-b63e-8e4730cf2886 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:10.743223 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:10.832488 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 960/1950] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:26:10 2022] POST /identity/v3/users => generated 291 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:10.834279 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f3104-1720-4b6c-b63e-8e4730cf2886 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:10.847789 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f3104-1720-4b6c-b63e-8e4730cf2886 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:10.854497 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3543e-90c9-43c0-9ec6-c4054b0eca43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:10.855496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3543e-90c9-43c0-9ec6-c4054b0eca43 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:10.855871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3543e-90c9-43c0-9ec6-c4054b0eca43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:10.856592 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3543e-90c9-43c0-9ec6-c4054b0eca43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:10.857687 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c3543e-90c9-43c0-9ec6-c4054b0eca43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:10.859620 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c3543e-90c9-43c0-9ec6-c4054b0eca43 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:10.873670 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 961/1951] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:26:10.877953 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3543e-90c9-43c0-9ec6-c4054b0eca43 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:10.898317 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9b2b3885-ac6a-4550-996e-e1f9a5d0e7e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cfe2d28f8e94d63a334ec53c100e889'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:10.901800 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:10.906790 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9b2b3885-ac6a-4550-996e-e1f9a5d0e7e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cfe2d28f8e94d63a334ec53c100e889'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:10.907582 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9b2b3885-ac6a-4550-996e-e1f9a5d0e7e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cfe2d28f8e94d63a334ec53c100e889'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:10.907736 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3543e-90c9-43c0-9ec6-c4054b0eca43 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:10.910090 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9b2b3885-ac6a-4550-996e-e1f9a5d0e7e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cfe2d28f8e94d63a334ec53c100e889'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:10.915162 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdefd5a-839d-4f7f-b64b-e5f569ca5e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:10.915796 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdefd5a-839d-4f7f-b64b-e5f569ca5e4f admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:10.916153 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdefd5a-839d-4f7f-b64b-e5f569ca5e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:10.916518 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdefd5a-839d-4f7f-b64b-e5f569ca5e4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:10.917680 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdefd5a-839d-4f7f-b64b-e5f569ca5e4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:10.920179 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdefd5a-839d-4f7f-b64b-e5f569ca5e4f admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:10.922775 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9b2b3885-ac6a-4550-996e-e1f9a5d0e7e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cfe2d28f8e94d63a334ec53c100e889'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:10.941780 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 962/1952] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:26:10.941780 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdefd5a-839d-4f7f-b64b-e5f569ca5e4f admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:10.941780 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-fbdefd5a-839d-4f7f-b64b-e5f569ca5e4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:10.962085 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdefd5a-839d-4f7f-b64b-e5f569ca5e4f admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:10.968115 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec972e1-895a-4916-9cf1-79d256332c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:10.969027 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec972e1-895a-4916-9cf1-79d256332c84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:10.969308 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec972e1-895a-4916-9cf1-79d256332c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:10.969979 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec972e1-895a-4916-9cf1-79d256332c84 admin admin] PATH_INFO: `/v3/projects/6a671d549796458eb41071e4752c1b73/users/c390661eb9e344b2824582ad6858fdf4/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:10.992780 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec972e1-895a-4916-9cf1-79d256332c84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a671d549796458eb41071e4752c1b73, user_id=c390661eb9e344b2824582ad6858fdf4, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:10.995239 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec972e1-895a-4916-9cf1-79d256332c84 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:10.998301 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 991/1953] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:26:10 2022] DELETE /identity/v3/users/4cfe2d28f8e94d63a334ec53c100e889 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:11.004773 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc89846-27cb-4241-9193-5792367217c9 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:11.005299 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc89846-27cb-4241-9193-5792367217c9 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:11.005551 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc89846-27cb-4241-9193-5792367217c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:11.030412 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-bdc89846-27cb-4241-9193-5792367217c9 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:11.032655 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-bdc89846-27cb-4241-9193-5792367217c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:11.132160 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 992/1954] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:11.142544 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc89846-27cb-4241-9193-5792367217c9 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:11.144639 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 963/1955] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:26:10 2022] PUT /identity/v3/projects/6a671d549796458eb41071e4752c1b73/users/c390661eb9e344b2824582ad6858fdf4/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:11.149890 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-bdc89846-27cb-4241-9193-5792367217c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:11.155801 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e95f8b-543a-4ef3-94d7-c2706aebda95 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:11.156081 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e95f8b-543a-4ef3-94d7-c2706aebda95 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:11.156610 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e95f8b-543a-4ef3-94d7-c2706aebda95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:11.187378 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-44e95f8b-543a-4ef3-94d7-c2706aebda95 None None] MFA Rules not processed for user `c390661eb9e344b2824582ad6858fdf4`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:11.187378 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-44e95f8b-543a-4ef3-94d7-c2706aebda95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:11.218869 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc89846-27cb-4241-9193-5792367217c9 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:11.236795 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:11.271202 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 964/1956] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:26:11.276053 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e95f8b-543a-4ef3-94d7-c2706aebda95 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:11.277625 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-44e95f8b-543a-4ef3-94d7-c2706aebda95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:11.309939 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935cf621-dfbf-490e-b482-ec35fdeb5efb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:11.311018 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935cf621-dfbf-490e-b482-ec35fdeb5efb service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:11.311301 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935cf621-dfbf-490e-b482-ec35fdeb5efb service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:11.311576 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935cf621-dfbf-490e-b482-ec35fdeb5efb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:11.376115 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:11.382516 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e95f8b-543a-4ef3-94d7-c2706aebda95 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:11.388391 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f252e80-5766-4e58-b26f-da997c443499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:11.389079 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f252e80-5766-4e58-b26f-da997c443499 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:11.389422 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f252e80-5766-4e58-b26f-da997c443499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:11.389703 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f252e80-5766-4e58-b26f-da997c443499 admin admin] PATH_INFO: `/v3/projects/a24923573225436a99c422f9bdbedabd` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:11.390905 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f252e80-5766-4e58-b26f-da997c443499 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a24923573225436a99c422f9bdbedabd)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:11.393171 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f252e80-5766-4e58-b26f-da997c443499 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:11.403183 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935cf621-dfbf-490e-b482-ec35fdeb5efb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:11.405823 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935cf621-dfbf-490e-b482-ec35fdeb5efb service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:11.419269 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 993/1957] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 14:26:11.420271 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935cf621-dfbf-490e-b482-ec35fdeb5efb service neutron] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:11.434223 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935cf621-dfbf-490e-b482-ec35fdeb5efb service neutron] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:11.441694 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8cc0b0-0a56-438b-ad0d-66aaad45e9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:11.442595 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8cc0b0-0a56-438b-ad0d-66aaad45e9e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:11.442881 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8cc0b0-0a56-438b-ad0d-66aaad45e9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:11.443302 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8cc0b0-0a56-438b-ad0d-66aaad45e9e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:11.444029 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8cc0b0-0a56-438b-ad0d-66aaad45e9e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:11.446023 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8cc0b0-0a56-438b-ad0d-66aaad45e9e2 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:11.534223 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 994/1958] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:26:11 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:11.538900 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8cc0b0-0a56-438b-ad0d-66aaad45e9e2 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:11.545068 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-2f252e80-5766-4e58-b26f-da997c443499 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a24923573225436a99c422f9bdbedabd', 'request_id': 'req-2f252e80-5766-4e58-b26f-da997c443499'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:11.552051 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-2f252e80-5766-4e58-b26f-da997c443499 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a24923573225436a99c422f9bdbedabd', 'request_id': 'req-2f252e80-5766-4e58-b26f-da997c443499'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:11.601394 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8cc0b0-0a56-438b-ad0d-66aaad45e9e2 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:11.606228 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23484b37-71df-4817-9634-1b69b5959a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:11.606869 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23484b37-71df-4817-9634-1b69b5959a1c admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:11.607195 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23484b37-71df-4817-9634-1b69b5959a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:11.607470 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23484b37-71df-4817-9634-1b69b5959a1c admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:11.608128 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23484b37-71df-4817-9634-1b69b5959a1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:11.610101 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23484b37-71df-4817-9634-1b69b5959a1c admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:11.637277 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:11.656705 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-2f252e80-5766-4e58-b26f-da997c443499 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a24923573225436a99c422f9bdbedabd', 'request_id': 'req-2f252e80-5766-4e58-b26f-da997c443499'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:11.670073 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-2f252e80-5766-4e58-b26f-da997c443499 admin admin] The token cache is being invalidate because project a24923573225436a99c422f9bdbedabd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:26:11.670359 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-2f252e80-5766-4e58-b26f-da997c443499 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:11.671042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-2f252e80-5766-4e58-b26f-da997c443499 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:11.672649 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 965/1959] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:26:11 2022] DELETE /identity/v3/projects/a24923573225436a99c422f9bdbedabd => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:11.681142 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf8b58-6bbf-4274-9013-d715bcf8e263 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:11.681640 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf8b58-6bbf-4274-9013-d715bcf8e263 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:11.686181 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf8b58-6bbf-4274-9013-d715bcf8e263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:11.712223 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-3edf8b58-6bbf-4274-9013-d715bcf8e263 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:11.715137 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-3edf8b58-6bbf-4274-9013-d715bcf8e263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:11.726250 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 966/1960] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:11.735329 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c04d7c-7c95-4c40-8dd4-6722c656679d None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:11.738301 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c04d7c-7c95-4c40-8dd4-6722c656679d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:11.745685 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 995/1961] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:26:11 2022] POST /identity/v3/users => generated 298 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:26:11.745963 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c04d7c-7c95-4c40-8dd4-6722c656679d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:11.753426 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23484b37-71df-4817-9634-1b69b5959a1c admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:11.755461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-23484b37-71df-4817-9634-1b69b5959a1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:11.776875 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23484b37-71df-4817-9634-1b69b5959a1c admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:11.779607 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-f2c04d7c-7c95-4c40-8dd4-6722c656679d None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:11.781318 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f2c04d7c-7c95-4c40-8dd4-6722c656679d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:11.790539 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d1b594-5d4e-468b-aecf-2d424f77b679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:11.791289 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1b594-5d4e-468b-aecf-2d424f77b679 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:11.791455 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1b594-5d4e-468b-aecf-2d424f77b679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:11.791725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1b594-5d4e-468b-aecf-2d424f77b679 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:11.792532 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d1b594-5d4e-468b-aecf-2d424f77b679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:11.794734 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d1b594-5d4e-468b-aecf-2d424f77b679 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:11.804705 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 967/1962] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:11.806189 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c04d7c-7c95-4c40-8dd4-6722c656679d None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:11.823020 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 996/1963] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:26:11.827147 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d1b594-5d4e-468b-aecf-2d424f77b679 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:11.838402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d1b594-5d4e-468b-aecf-2d424f77b679 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:11.841981 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c04d7c-7c95-4c40-8dd4-6722c656679d None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:11.844156 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91cfbe6-0dbe-4008-9a66-efc4115cc9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:11.844914 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91cfbe6-0dbe-4008-9a66-efc4115cc9c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:11.845247 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91cfbe6-0dbe-4008-9a66-efc4115cc9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:11.845527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91cfbe6-0dbe-4008-9a66-efc4115cc9c0 admin admin] PATH_INFO: `/v3/projects/990173ac30f54cf5876b635ff3cfb184/users/4a7f0c09aeaf427085ecf514502afd65/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:11.848091 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e9cbed-693c-4c7e-9a32-b3816c9ce3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:11.848760 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9cbed-693c-4c7e-9a32-b3816c9ce3b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:11.849036 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9cbed-693c-4c7e-9a32-b3816c9ce3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:11.849369 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9cbed-693c-4c7e-9a32-b3816c9ce3b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:11.850204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e9cbed-693c-4c7e-9a32-b3816c9ce3b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:26:11.850505 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e9cbed-693c-4c7e-9a32-b3816c9ce3b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:11.852693 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e9cbed-693c-4c7e-9a32-b3816c9ce3b9 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:11.865820 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91cfbe6-0dbe-4008-9a66-efc4115cc9c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=990173ac30f54cf5876b635ff3cfb184, user_id=4a7f0c09aeaf427085ecf514502afd65, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:11.867342 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 968/1964] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:26:11.872935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91cfbe6-0dbe-4008-9a66-efc4115cc9c0 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:11.873024 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e9cbed-693c-4c7e-9a32-b3816c9ce3b9 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:11.893198 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e9cbed-693c-4c7e-9a32-b3816c9ce3b9 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:11.899158 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376214bd-4c4d-419d-ad69-347fe432ffee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:11.899903 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376214bd-4c4d-419d-ad69-347fe432ffee admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:11.900275 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376214bd-4c4d-419d-ad69-347fe432ffee admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:11.900778 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376214bd-4c4d-419d-ad69-347fe432ffee admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:11.901541 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376214bd-4c4d-419d-ad69-347fe432ffee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:11.903546 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376214bd-4c4d-419d-ad69-347fe432ffee admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:11.967686 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 997/1965] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:26:11 2022] PUT /identity/v3/projects/990173ac30f54cf5876b635ff3cfb184/users/4a7f0c09aeaf427085ecf514502afd65/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:11.967686 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91cfbe6-0dbe-4008-9a66-efc4115cc9c0 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:11.968088 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-f91cfbe6-0dbe-4008-9a66-efc4115cc9c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:11.994574 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 969/1966] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:26:11 2022] POST /identity/v3/projects => generated 381 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:11.995500 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376214bd-4c4d-419d-ad69-347fe432ffee admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:12.035254 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91cfbe6-0dbe-4008-9a66-efc4115cc9c0 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:12.057754 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376214bd-4c4d-419d-ad69-347fe432ffee admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:12.062616 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b0852-7b03-4f95-82c5-827a916b58bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:12.063241 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b0852-7b03-4f95-82c5-827a916b58bc admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:12.064770 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b0852-7b03-4f95-82c5-827a916b58bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:12.064770 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b0852-7b03-4f95-82c5-827a916b58bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:12.064770 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4b0852-7b03-4f95-82c5-827a916b58bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:12.067305 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4b0852-7b03-4f95-82c5-827a916b58bc admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:12.077632 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 970/1967] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:26:12.078319 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b0852-7b03-4f95-82c5-827a916b58bc admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:12.094268 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b0852-7b03-4f95-82c5-827a916b58bc admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:12.100803 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8f681-5c57-41d5-81d0-c5d47e000c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:12.101901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8f681-5c57-41d5-81d0-c5d47e000c96 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:12.102255 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8f681-5c57-41d5-81d0-c5d47e000c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:12.102604 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8f681-5c57-41d5-81d0-c5d47e000c96 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:12.104781 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec8f681-5c57-41d5-81d0-c5d47e000c96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:12.110349 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec8f681-5c57-41d5-81d0-c5d47e000c96 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:12.110745 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4ae93-e18c-4e3f-91a2-f14e57e8bba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:12.111686 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4ae93-e18c-4e3f-91a2-f14e57e8bba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:12.112060 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4ae93-e18c-4e3f-91a2-f14e57e8bba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:12.112517 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4ae93-e18c-4e3f-91a2-f14e57e8bba5 admin admin] PATH_INFO: `/v3/projects/567dbab3a66a489aa09e15c582f73c9f` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:12.114388 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4ae93-e18c-4e3f-91a2-f14e57e8bba5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=567dbab3a66a489aa09e15c582f73c9f)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:12.116441 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4ae93-e18c-4e3f-91a2-f14e57e8bba5 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:12.227533 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 971/1968] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:26:12 2022] POST /identity/v3/users => generated 291 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:12.231656 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8f681-5c57-41d5-81d0-c5d47e000c96 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:12.242454 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8f681-5c57-41d5-81d0-c5d47e000c96 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:12.247159 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb971ce-edc5-4726-a53c-c1d666da4dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:12.247962 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb971ce-edc5-4726-a53c-c1d666da4dc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:12.248247 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb971ce-edc5-4726-a53c-c1d666da4dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:12.248565 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb971ce-edc5-4726-a53c-c1d666da4dc0 admin admin] PATH_INFO: `/v3/projects/990173ac30f54cf5876b635ff3cfb184/users/4a7f0c09aeaf427085ecf514502afd65/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:12.250213 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:12.253125 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb971ce-edc5-4726-a53c-c1d666da4dc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=990173ac30f54cf5876b635ff3cfb184, user_id=4a7f0c09aeaf427085ecf514502afd65, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:12.254102 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb971ce-edc5-4726-a53c-c1d666da4dc0 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:12.266903 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 972/1969] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:26:12 2022] PUT /identity/v3/projects/990173ac30f54cf5876b635ff3cfb184/users/4a7f0c09aeaf427085ecf514502afd65/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:12.267631 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb971ce-edc5-4726-a53c-c1d666da4dc0 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:12.293042 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-2ac4ae93-e18c-4e3f-91a2-f14e57e8bba5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '567dbab3a66a489aa09e15c582f73c9f', 'request_id': 'req-2ac4ae93-e18c-4e3f-91a2-f14e57e8bba5'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:12.294673 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-2ac4ae93-e18c-4e3f-91a2-f14e57e8bba5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '567dbab3a66a489aa09e15c582f73c9f', 'request_id': 'req-2ac4ae93-e18c-4e3f-91a2-f14e57e8bba5'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:12.302610 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-2ac4ae93-e18c-4e3f-91a2-f14e57e8bba5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '567dbab3a66a489aa09e15c582f73c9f', 'request_id': 'req-2ac4ae93-e18c-4e3f-91a2-f14e57e8bba5'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:12.369322 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb971ce-edc5-4726-a53c-c1d666da4dc0 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:12.374183 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2fc3d-4cd0-4258-9cad-347e7889e71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:12.374813 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2fc3d-4cd0-4258-9cad-347e7889e71f admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:12.375116 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2fc3d-4cd0-4258-9cad-347e7889e71f admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:12.375419 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2fc3d-4cd0-4258-9cad-347e7889e71f admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:12.376120 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2fc3d-4cd0-4258-9cad-347e7889e71f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:12.377939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2fc3d-4cd0-4258-9cad-347e7889e71f admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:12.389206 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 973/1970] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:26:12.390694 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798fb25-8bcb-488d-a422-5fa438292bba None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:12.390991 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798fb25-8bcb-488d-a422-5fa438292bba None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:12.391269 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798fb25-8bcb-488d-a422-5fa438292bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:12.400932 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-2ac4ae93-e18c-4e3f-91a2-f14e57e8bba5 admin admin] The token cache is being invalidate because project 567dbab3a66a489aa09e15c582f73c9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:26:12.401738 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-2ac4ae93-e18c-4e3f-91a2-f14e57e8bba5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:12.402445 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-2ac4ae93-e18c-4e3f-91a2-f14e57e8bba5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:12.407796 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 998/1971] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:26:11 2022] DELETE /identity/v3/projects/567dbab3a66a489aa09e15c582f73c9f => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:12.411629 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4ae93-e18c-4e3f-91a2-f14e57e8bba5 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:12.413104 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-2ac4ae93-e18c-4e3f-91a2-f14e57e8bba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:12.451334 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-3798fb25-8bcb-488d-a422-5fa438292bba None None] MFA Rules not processed for user `4a7f0c09aeaf427085ecf514502afd65`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:12.453303 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-3798fb25-8bcb-488d-a422-5fa438292bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:12.482358 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:12.502371 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4ae93-e18c-4e3f-91a2-f14e57e8bba5 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:12.509519 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe2295-a35c-4dc7-a0a3-37db1a324544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:12.510216 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe2295-a35c-4dc7-a0a3-37db1a324544 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:12.510496 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe2295-a35c-4dc7-a0a3-37db1a324544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:12.510858 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe2295-a35c-4dc7-a0a3-37db1a324544 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:12.511644 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fe2295-a35c-4dc7-a0a3-37db1a324544 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:12.513355 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fe2295-a35c-4dc7-a0a3-37db1a324544 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:12.528931 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 999/1972] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:26:12.530354 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe2295-a35c-4dc7-a0a3-37db1a324544 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:12.532105 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-a0fe2295-a35c-4dc7-a0a3-37db1a324544 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:12.540309 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 974/1973] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:12 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:12.541341 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3798fb25-8bcb-488d-a422-5fa438292bba None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:12.546801 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe2295-a35c-4dc7-a0a3-37db1a324544 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:12.553172 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d958361-9759-45ff-8461-e26097f23627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:12.553831 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d958361-9759-45ff-8461-e26097f23627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:12.554116 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d958361-9759-45ff-8461-e26097f23627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:12.554392 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d958361-9759-45ff-8461-e26097f23627 admin admin] PATH_INFO: `/v3/users/561ec5349565401fb2f758af3314b755` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:12.556013 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d958361-9759-45ff-8461-e26097f23627 admin admin] RBAC: Authorizing `identity:delete_user(user_id=561ec5349565401fb2f758af3314b755)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:12.558131 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d958361-9759-45ff-8461-e26097f23627 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:12.558946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3798fb25-8bcb-488d-a422-5fa438292bba None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:12.564534 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1898b-d499-4179-a6dd-9444e84df5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:12.565215 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1898b-d499-4179-a6dd-9444e84df5c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:12.566176 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1898b-d499-4179-a6dd-9444e84df5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:12.566437 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1898b-d499-4179-a6dd-9444e84df5c6 admin admin] PATH_INFO: `/v3/projects/4befd2689cb84279bc430d344aba0ec4/users/204af87dd73e45bda2e0cd4c98c92328/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:12.588166 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f1898b-d499-4179-a6dd-9444e84df5c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4befd2689cb84279bc430d344aba0ec4, user_id=204af87dd73e45bda2e0cd4c98c92328, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:12.590848 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f1898b-d499-4179-a6dd-9444e84df5c6 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:12.662017 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 975/1974] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:26:12 2022] PUT /identity/v3/projects/4befd2689cb84279bc430d344aba0ec4/users/204af87dd73e45bda2e0cd4c98c92328/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:12.671152 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1898b-d499-4179-a6dd-9444e84df5c6 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:12.672029 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-64f1898b-d499-4179-a6dd-9444e84df5c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:12.728297 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1898b-d499-4179-a6dd-9444e84df5c6 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:12.732080 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd6c7f4-ed00-4bc9-8ece-92cb3947274e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:12.732725 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6c7f4-ed00-4bc9-8ece-92cb3947274e service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:12.733019 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6c7f4-ed00-4bc9-8ece-92cb3947274e service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:12.733273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6c7f4-ed00-4bc9-8ece-92cb3947274e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:12.781376 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-0d958361-9759-45ff-8461-e26097f23627 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '561ec5349565401fb2f758af3314b755'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:12.789760 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-0d958361-9759-45ff-8461-e26097f23627 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '561ec5349565401fb2f758af3314b755'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:12.790611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-0d958361-9759-45ff-8461-e26097f23627 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '561ec5349565401fb2f758af3314b755'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:12.791234 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-0d958361-9759-45ff-8461-e26097f23627 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '561ec5349565401fb2f758af3314b755'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:12.798649 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd6c7f4-ed00-4bc9-8ece-92cb3947274e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:12.800798 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd6c7f4-ed00-4bc9-8ece-92cb3947274e service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:12.805092 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-0d958361-9759-45ff-8461-e26097f23627 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '561ec5349565401fb2f758af3314b755'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:12.810765 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 976/1975] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:12 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:26:12.812041 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b5178-031d-49e0-8694-1ac405f41f76 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:12.812329 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b5178-031d-49e0-8694-1ac405f41f76 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:12.812708 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b5178-031d-49e0-8694-1ac405f41f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:12.842026 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-9f7b5178-031d-49e0-8694-1ac405f41f76 None None] MFA Rules not processed for user `204af87dd73e45bda2e0cd4c98c92328`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:12.843230 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-9f7b5178-031d-49e0-8694-1ac405f41f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:12.903419 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1000/1976] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:26:12 2022] DELETE /identity/v3/users/561ec5349565401fb2f758af3314b755 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:12.914639 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 977/1977] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:12.919131 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d958361-9759-45ff-8461-e26097f23627 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:12.920750 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-0d958361-9759-45ff-8461-e26097f23627 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:13.015709 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b5178-031d-49e0-8694-1ac405f41f76 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:13.017546 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-9f7b5178-031d-49e0-8694-1ac405f41f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:13.026427 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d958361-9759-45ff-8461-e26097f23627 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:13.033917 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea59d5-f537-4a38-86a1-7deede7293d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:13.035356 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea59d5-f537-4a38-86a1-7deede7293d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:13.035823 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea59d5-f537-4a38-86a1-7deede7293d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:13.041031 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea59d5-f537-4a38-86a1-7deede7293d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:13.041031 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ea59d5-f537-4a38-86a1-7deede7293d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:13.041031 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ea59d5-f537-4a38-86a1-7deede7293d7 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:13.048126 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b5178-031d-49e0-8694-1ac405f41f76 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:13.119453 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692c262-6f9b-4e46-ae80-23b6f387f064 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:13.120022 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692c262-6f9b-4e46-ae80-23b6f387f064 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:13.120881 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692c262-6f9b-4e46-ae80-23b6f387f064 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:13.120881 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692c262-6f9b-4e46-ae80-23b6f387f064 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:13.122936 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-0692c262-6f9b-4e46-ae80-23b6f387f064 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:13.171372 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1001/1978] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:26:12 2022] POST /identity/v3/projects => generated 379 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:26:13.172845 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea59d5-f537-4a38-86a1-7deede7293d7 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:13.233086 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0692c262-6f9b-4e46-ae80-23b6f387f064 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:13.235445 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0692c262-6f9b-4e46-ae80-23b6f387f064 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:13.258379 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 978/1979] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:13 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 14:26:13.274375 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea59d5-f537-4a38-86a1-7deede7293d7 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:13.283017 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3463a9e-bffe-4817-983c-ca206d776062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:13.283667 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3463a9e-bffe-4817-983c-ca206d776062 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:13.283950 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3463a9e-bffe-4817-983c-ca206d776062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:13.284269 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3463a9e-bffe-4817-983c-ca206d776062 admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:13.285031 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3463a9e-bffe-4817-983c-ca206d776062 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:13.290831 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3463a9e-bffe-4817-983c-ca206d776062 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:13.423765 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1002/1980] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:26:13 2022] POST /identity/v3/users => generated 296 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:13.428662 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692c262-6f9b-4e46-ae80-23b6f387f064 service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:13.446140 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692c262-6f9b-4e46-ae80-23b6f387f064 service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:13.450445 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ba3ee-7ac6-4402-bfb0-776a9a40dda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:13.451060 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ba3ee-7ac6-4402-bfb0-776a9a40dda3 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:13.451346 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ba3ee-7ac6-4402-bfb0-776a9a40dda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:13.451607 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ba3ee-7ac6-4402-bfb0-776a9a40dda3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:13.452291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ba3ee-7ac6-4402-bfb0-776a9a40dda3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:13.453930 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ba3ee-7ac6-4402-bfb0-776a9a40dda3 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:13.466669 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 979/1981] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:26:13.470771 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3463a9e-bffe-4817-983c-ca206d776062 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:13.485936 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3463a9e-bffe-4817-983c-ca206d776062 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:13.494112 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ba3ee-7ac6-4402-bfb0-776a9a40dda3 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:13.494525 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb849ed-df07-4be8-8a62-42df96c4d48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:13.496580 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb849ed-df07-4be8-8a62-42df96c4d48b admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:13.496580 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb849ed-df07-4be8-8a62-42df96c4d48b admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:13.496580 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb849ed-df07-4be8-8a62-42df96c4d48b admin admin] PATH_INFO: `/v3/projects/c0f10f45a4e7452692cb2bab94adbe0b/users/e741838ee19c4c719f2eea660a58119b/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:13.497964 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-1a7ba3ee-7ac6-4402-bfb0-776a9a40dda3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:13.526672 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ba3ee-7ac6-4402-bfb0-776a9a40dda3 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:13.535614 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4caab45-9e59-4bcd-8868-2e4bc1985fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:13.535614 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4caab45-9e59-4bcd-8868-2e4bc1985fe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:13.535614 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4caab45-9e59-4bcd-8868-2e4bc1985fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:13.535614 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4caab45-9e59-4bcd-8868-2e4bc1985fe9 admin admin] PATH_INFO: `/v3/projects/01ddb9a36c2b4d7786644b4c04efc1e5` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:13.536089 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4caab45-9e59-4bcd-8868-2e4bc1985fe9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01ddb9a36c2b4d7786644b4c04efc1e5)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:13.536160 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb849ed-df07-4be8-8a62-42df96c4d48b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0f10f45a4e7452692cb2bab94adbe0b, user_id=e741838ee19c4c719f2eea660a58119b, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:13.537343 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4caab45-9e59-4bcd-8868-2e4bc1985fe9 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:13.538238 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb849ed-df07-4be8-8a62-42df96c4d48b admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:13.629070 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1003/1982] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:26:13 2022] PUT /identity/v3/projects/c0f10f45a4e7452692cb2bab94adbe0b/users/e741838ee19c4c719f2eea660a58119b/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:13.633555 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb849ed-df07-4be8-8a62-42df96c4d48b admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:13.672583 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f4caab45-9e59-4bcd-8868-2e4bc1985fe9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01ddb9a36c2b4d7786644b4c04efc1e5', 'request_id': 'req-f4caab45-9e59-4bcd-8868-2e4bc1985fe9'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:13.685754 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f4caab45-9e59-4bcd-8868-2e4bc1985fe9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01ddb9a36c2b4d7786644b4c04efc1e5', 'request_id': 'req-f4caab45-9e59-4bcd-8868-2e4bc1985fe9'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:13.709923 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb849ed-df07-4be8-8a62-42df96c4d48b admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:13.756981 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f4caab45-9e59-4bcd-8868-2e4bc1985fe9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01ddb9a36c2b4d7786644b4c04efc1e5', 'request_id': 'req-f4caab45-9e59-4bcd-8868-2e4bc1985fe9'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:13.768403 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca4471-04bd-4a8f-be42-33730bc0f876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:13.770226 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca4471-04bd-4a8f-be42-33730bc0f876 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:13.770496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f4caab45-9e59-4bcd-8868-2e4bc1985fe9 admin admin] The token cache is being invalidate because project 01ddb9a36c2b4d7786644b4c04efc1e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:26:13.770607 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca4471-04bd-4a8f-be42-33730bc0f876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:13.770838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca4471-04bd-4a8f-be42-33730bc0f876 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:13.770951 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f4caab45-9e59-4bcd-8868-2e4bc1985fe9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:13.771509 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f4caab45-9e59-4bcd-8868-2e4bc1985fe9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:13.771608 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca4471-04bd-4a8f-be42-33730bc0f876 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:13.773320 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 980/1983] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:26:13 2022] DELETE /identity/v3/projects/01ddb9a36c2b4d7786644b4c04efc1e5 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:13.773459 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca4471-04bd-4a8f-be42-33730bc0f876 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:13.778158 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af596bd-ce49-4418-8e56-4d2f00985f4d None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:13.778454 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af596bd-ce49-4418-8e56-4d2f00985f4d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:13.778733 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af596bd-ce49-4418-8e56-4d2f00985f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:13.788925 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1004/1984] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:26:13.790057 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca4471-04bd-4a8f-be42-33730bc0f876 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:13.791733 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-20ca4471-04bd-4a8f-be42-33730bc0f876 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:13.803062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-4af596bd-ce49-4418-8e56-4d2f00985f4d None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:13.806634 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca4471-04bd-4a8f-be42-33730bc0f876 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:13.806716 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-4af596bd-ce49-4418-8e56-4d2f00985f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:13.809224 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea149a-0a90-4911-8950-a2af5d74f17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:13.809951 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea149a-0a90-4911-8950-a2af5d74f17f admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:13.810231 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea149a-0a90-4911-8950-a2af5d74f17f admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:13.810501 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea149a-0a90-4911-8950-a2af5d74f17f admin admin] PATH_INFO: `/v3/projects/c0f10f45a4e7452692cb2bab94adbe0b/users/e741838ee19c4c719f2eea660a58119b/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:13.812748 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea149a-0a90-4911-8950-a2af5d74f17f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0f10f45a4e7452692cb2bab94adbe0b, user_id=e741838ee19c4c719f2eea660a58119b, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:13.821706 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea149a-0a90-4911-8950-a2af5d74f17f admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:13.821876 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 981/1985] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:26:13.821876 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af596bd-ce49-4418-8e56-4d2f00985f4d None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:13.821876 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-4af596bd-ce49-4418-8e56-4d2f00985f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:13.835266 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af596bd-ce49-4418-8e56-4d2f00985f4d None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:13.836829 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1005/1986] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:26:13 2022] PUT /identity/v3/projects/c0f10f45a4e7452692cb2bab94adbe0b/users/e741838ee19c4c719f2eea660a58119b/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:26:13.838267 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66c42a-2cd6-404d-a685-42b1b338e7f8 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:13.838560 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66c42a-2cd6-404d-a685-42b1b338e7f8 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:13.838839 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66c42a-2cd6-404d-a685-42b1b338e7f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:13.861053 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-4c66c42a-2cd6-404d-a685-42b1b338e7f8 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:13.862334 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-4c66c42a-2cd6-404d-a685-42b1b338e7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:13.899090 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8fb0a-61d5-460f-9016-97a038939e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:13.899733 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8fb0a-61d5-460f-9016-97a038939e60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:13.900010 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8fb0a-61d5-460f-9016-97a038939e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:13.900281 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8fb0a-61d5-460f-9016-97a038939e60 admin admin] PATH_INFO: `/v3/users/c390661eb9e344b2824582ad6858fdf4` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:13.901471 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca8fb0a-61d5-460f-9016-97a038939e60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c390661eb9e344b2824582ad6858fdf4)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:13.903368 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca8fb0a-61d5-460f-9016-97a038939e60 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:13.915885 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:13.927082 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1006/1987] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:13.928457 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695645b-9117-4dd1-a275-b2e96e36275a None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:13.928745 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695645b-9117-4dd1-a275-b2e96e36275a None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:13.929014 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695645b-9117-4dd1-a275-b2e96e36275a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:13.956572 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-b695645b-9117-4dd1-a275-b2e96e36275a None None] MFA Rules not processed for user `e741838ee19c4c719f2eea660a58119b`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:13.957840 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-b695645b-9117-4dd1-a275-b2e96e36275a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:14.025049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1007/1988] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:13 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:14.025883 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695645b-9117-4dd1-a275-b2e96e36275a None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:14.037529 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695645b-9117-4dd1-a275-b2e96e36275a None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:14.042094 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8334ef7-fb92-4583-ac2c-e3c13eb01173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:14.042716 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8334ef7-fb92-4583-ac2c-e3c13eb01173 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:14.042973 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8334ef7-fb92-4583-ac2c-e3c13eb01173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:14.043276 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8334ef7-fb92-4583-ac2c-e3c13eb01173 admin admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:14.043924 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8334ef7-fb92-4583-ac2c-e3c13eb01173 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:26:14.044223 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8334ef7-fb92-4583-ac2c-e3c13eb01173 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:14.045820 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8334ef7-fb92-4583-ac2c-e3c13eb01173 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:14.057827 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1008/1989] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:26:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:26:14.058599 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8334ef7-fb92-4583-ac2c-e3c13eb01173 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:14.060180 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-a8334ef7-fb92-4583-ac2c-e3c13eb01173 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:14.116580 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8334ef7-fb92-4583-ac2c-e3c13eb01173 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:14.120573 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c950d-2da6-45ec-a87d-052d204e7d41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:14.121184 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c950d-2da6-45ec-a87d-052d204e7d41 service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:14.121488 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c950d-2da6-45ec-a87d-052d204e7d41 service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:14.121756 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c950d-2da6-45ec-a87d-052d204e7d41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:14.131259 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7c950d-2da6-45ec-a87d-052d204e7d41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:14.132709 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7c950d-2da6-45ec-a87d-052d204e7d41 service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:14.141954 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1009/1990] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:14 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:26:14.142704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c950d-2da6-45ec-a87d-052d204e7d41 service cinder] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:14.152991 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c950d-2da6-45ec-a87d-052d204e7d41 service cinder] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:14.157583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f001a5af-31da-42fc-a19e-028bb9dbe236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:14.158175 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001a5af-31da-42fc-a19e-028bb9dbe236 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:14.162264 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001a5af-31da-42fc-a19e-028bb9dbe236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:14.162264 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001a5af-31da-42fc-a19e-028bb9dbe236 admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:14.162264 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f001a5af-31da-42fc-a19e-028bb9dbe236 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:14.162264 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f001a5af-31da-42fc-a19e-028bb9dbe236 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:14.182972 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:14.235416 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8ca8fb0a-61d5-460f-9016-97a038939e60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c390661eb9e344b2824582ad6858fdf4'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:14.251050 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8ca8fb0a-61d5-460f-9016-97a038939e60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c390661eb9e344b2824582ad6858fdf4'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:14.304152 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1010/1991] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:26:14 2022] POST /identity/v3/projects => generated 379 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:14.308125 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f001a5af-31da-42fc-a19e-028bb9dbe236 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:14.325344 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8ca8fb0a-61d5-460f-9016-97a038939e60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c390661eb9e344b2824582ad6858fdf4'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:14.325980 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8ca8fb0a-61d5-460f-9016-97a038939e60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c390661eb9e344b2824582ad6858fdf4'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:14.354315 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8ca8fb0a-61d5-460f-9016-97a038939e60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c390661eb9e344b2824582ad6858fdf4'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:14.368335 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 982/1992] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:26:13 2022] DELETE /identity/v3/users/c390661eb9e344b2824582ad6858fdf4 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:26:14.372486 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a7a43-51c0-4903-a5c8-473c7fb28da9 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:14.372814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a7a43-51c0-4903-a5c8-473c7fb28da9 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:14.373042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a7a43-51c0-4903-a5c8-473c7fb28da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:14.399051 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-425a7a43-51c0-4903-a5c8-473c7fb28da9 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:14.400274 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-425a7a43-51c0-4903-a5c8-473c7fb28da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:14.410392 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f001a5af-31da-42fc-a19e-028bb9dbe236 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:14.498110 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc23a35-b359-4d90-a3f1-f9bf91a03dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:14.498824 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc23a35-b359-4d90-a3f1-f9bf91a03dd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:14.499291 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc23a35-b359-4d90-a3f1-f9bf91a03dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:14.499575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc23a35-b359-4d90-a3f1-f9bf91a03dd3 admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:14.503620 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc23a35-b359-4d90-a3f1-f9bf91a03dd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:14.504390 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc23a35-b359-4d90-a3f1-f9bf91a03dd3 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:14.508654 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 983/1993] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:14.509514 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a7a43-51c0-4903-a5c8-473c7fb28da9 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:14.510885 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-425a7a43-51c0-4903-a5c8-473c7fb28da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:14.541272 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a7a43-51c0-4903-a5c8-473c7fb28da9 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:14.546800 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a408c-507e-4780-ae6f-58a3f8b4de85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:14.547453 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a408c-507e-4780-ae6f-58a3f8b4de85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:14.547731 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a408c-507e-4780-ae6f-58a3f8b4de85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:14.548167 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a408c-507e-4780-ae6f-58a3f8b4de85 admin admin] PATH_INFO: `/v3/users/e6736c8f94254a8897146abedb29593c` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:14.549959 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7a408c-507e-4780-ae6f-58a3f8b4de85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6736c8f94254a8897146abedb29593c)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:14.553602 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7a408c-507e-4780-ae6f-58a3f8b4de85 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:14.619714 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1011/1994] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:26:14 2022] POST /identity/v3/users => generated 290 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:26:14.620645 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc23a35-b359-4d90-a3f1-f9bf91a03dd3 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:14.622377 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-adc23a35-b359-4d90-a3f1-f9bf91a03dd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:14.688394 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc23a35-b359-4d90-a3f1-f9bf91a03dd3 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:14.693010 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71781056-84e3-4fda-8c2a-cf58abb905ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:14.693732 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71781056-84e3-4fda-8c2a-cf58abb905ec service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:14.694063 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71781056-84e3-4fda-8c2a-cf58abb905ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:14.694394 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71781056-84e3-4fda-8c2a-cf58abb905ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:14.705327 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71781056-84e3-4fda-8c2a-cf58abb905ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:14.708126 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71781056-84e3-4fda-8c2a-cf58abb905ec service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:14.718866 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1012/1995] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:26:14.719773 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71781056-84e3-4fda-8c2a-cf58abb905ec service neutron] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:14.721526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-71781056-84e3-4fda-8c2a-cf58abb905ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:14.740672 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71781056-84e3-4fda-8c2a-cf58abb905ec service neutron] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:14.745609 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d5686-651c-47de-8fc6-c6b35ae270c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:14.746234 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d5686-651c-47de-8fc6-c6b35ae270c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:14.746748 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d5686-651c-47de-8fc6-c6b35ae270c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:14.746850 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d5686-651c-47de-8fc6-c6b35ae270c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:14.747486 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94d5686-651c-47de-8fc6-c6b35ae270c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:14.750103 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94d5686-651c-47de-8fc6-c6b35ae270c7 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:14.752817 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1c7a408c-507e-4780-ae6f-58a3f8b4de85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6736c8f94254a8897146abedb29593c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:14.764059 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1013/1996] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:26:14.765127 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d5686-651c-47de-8fc6-c6b35ae270c7 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:14.770122 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1c7a408c-507e-4780-ae6f-58a3f8b4de85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6736c8f94254a8897146abedb29593c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:14.775924 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d5686-651c-47de-8fc6-c6b35ae270c7 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:14.780843 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ccaebe-4f78-47f1-a1f4-de1e10e8fdff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:14.781443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ccaebe-4f78-47f1-a1f4-de1e10e8fdff admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:14.781725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ccaebe-4f78-47f1-a1f4-de1e10e8fdff admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:14.781997 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ccaebe-4f78-47f1-a1f4-de1e10e8fdff admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:14.784119 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ccaebe-4f78-47f1-a1f4-de1e10e8fdff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:14.784493 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ccaebe-4f78-47f1-a1f4-de1e10e8fdff admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:14.794778 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1014/1997] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:26:14.798772 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ccaebe-4f78-47f1-a1f4-de1e10e8fdff admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:14.812809 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ccaebe-4f78-47f1-a1f4-de1e10e8fdff admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:14.818827 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dacfb6b-12b8-4908-9f2d-d2f7f0a53832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:14.819665 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dacfb6b-12b8-4908-9f2d-d2f7f0a53832 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:14.819949 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dacfb6b-12b8-4908-9f2d-d2f7f0a53832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:14.820256 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dacfb6b-12b8-4908-9f2d-d2f7f0a53832 admin admin] PATH_INFO: `/v3/projects/f5f6f677923d479b8b9d2b2f7445f374/users/5226fb20ff3c437882f9b8dce2cb83c8/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:14.837891 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dacfb6b-12b8-4908-9f2d-d2f7f0a53832 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5f6f677923d479b8b9d2b2f7445f374, user_id=5226fb20ff3c437882f9b8dce2cb83c8, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:14.840661 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dacfb6b-12b8-4908-9f2d-d2f7f0a53832 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:14.879082 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1c7a408c-507e-4780-ae6f-58a3f8b4de85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6736c8f94254a8897146abedb29593c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:14.879681 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1c7a408c-507e-4780-ae6f-58a3f8b4de85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6736c8f94254a8897146abedb29593c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:14.887725 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1c7a408c-507e-4780-ae6f-58a3f8b4de85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6736c8f94254a8897146abedb29593c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:14.894478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 984/1998] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:26:14 2022] DELETE /identity/v3/users/e6736c8f94254a8897146abedb29593c => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:14.899358 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77daf18-f3e5-4417-a56e-bdcdc0437880 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:14.899586 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77daf18-f3e5-4417-a56e-bdcdc0437880 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:14.899858 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77daf18-f3e5-4417-a56e-bdcdc0437880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:14.902060 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:14.911850 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1015/1999] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:26:14 2022] PUT /identity/v3/projects/f5f6f677923d479b8b9d2b2f7445f374/users/5226fb20ff3c437882f9b8dce2cb83c8/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:26:14.923334 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d3f70-06c9-4e7d-83cf-a5757b641b8e None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:14.923768 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d3f70-06c9-4e7d-83cf-a5757b641b8e None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:14.924164 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d3f70-06c9-4e7d-83cf-a5757b641b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:14.933774 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-f77daf18-f3e5-4417-a56e-bdcdc0437880 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:14.934997 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f77daf18-f3e5-4417-a56e-bdcdc0437880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:14.956889 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-cc7d3f70-06c9-4e7d-83cf-a5757b641b8e None None] MFA Rules not processed for user `5226fb20ff3c437882f9b8dce2cb83c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:14.958285 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-cc7d3f70-06c9-4e7d-83cf-a5757b641b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:14.983018 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:15.012028 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 985/2000] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:26:15.023576 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77daf18-f3e5-4417-a56e-bdcdc0437880 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:15.025392 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f77daf18-f3e5-4417-a56e-bdcdc0437880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:15.026280 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1016/2001] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:26:15.030634 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7d3f70-06c9-4e7d-83cf-a5757b641b8e None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:15.031995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-cc7d3f70-06c9-4e7d-83cf-a5757b641b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:15.049210 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7d3f70-06c9-4e7d-83cf-a5757b641b8e None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:15.055112 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa9849-ff01-45fb-8e09-95b8db5c51c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:15.055918 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa9849-ff01-45fb-8e09-95b8db5c51c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:15.056295 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa9849-ff01-45fb-8e09-95b8db5c51c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:15.056995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa9849-ff01-45fb-8e09-95b8db5c51c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:15.057587 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa9849-ff01-45fb-8e09-95b8db5c51c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:15.059603 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa9849-ff01-45fb-8e09-95b8db5c51c5 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:15.094488 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77daf18-f3e5-4417-a56e-bdcdc0437880 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:15.105386 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd37e02d-188d-4790-ba92-f59bd7a14f79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:15.106401 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37e02d-188d-4790-ba92-f59bd7a14f79 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:15.107490 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37e02d-188d-4790-ba92-f59bd7a14f79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:15.107972 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37e02d-188d-4790-ba92-f59bd7a14f79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:15.119276 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd37e02d-188d-4790-ba92-f59bd7a14f79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:15.123983 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd37e02d-188d-4790-ba92-f59bd7a14f79 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:15.131596 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 986/2002] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:26:15.185614 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1017/2003] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:26:15 2022] POST /identity/v3/projects => generated 377 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:26:15.190187 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd37e02d-188d-4790-ba92-f59bd7a14f79 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:15.202667 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd37e02d-188d-4790-ba92-f59bd7a14f79 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:15.280724 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa9849-ff01-45fb-8e09-95b8db5c51c5 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:15.283008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-b9fa9849-ff01-45fb-8e09-95b8db5c51c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:15.298183 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a197c5b2-74f9-4dbe-a822-e2f89ab67091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:15.300670 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197c5b2-74f9-4dbe-a822-e2f89ab67091 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:15.300670 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197c5b2-74f9-4dbe-a822-e2f89ab67091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:15.300670 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197c5b2-74f9-4dbe-a822-e2f89ab67091 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:15.300670 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a197c5b2-74f9-4dbe-a822-e2f89ab67091 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:15.302611 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a197c5b2-74f9-4dbe-a822-e2f89ab67091 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:15.392392 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa9849-ff01-45fb-8e09-95b8db5c51c5 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:15.402762 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d4c863-485f-4cfd-8d62-4b575e345798 tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e741838ee19c4c719f2eea660a58119b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0f10f45a4e7452692cb2bab94adbe0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:15.403528 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4c863-485f-4cfd-8d62-4b575e345798 tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:15.404184 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4c863-485f-4cfd-8d62-4b575e345798 tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:15.404694 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4c863-485f-4cfd-8d62-4b575e345798 tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:15.405365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d4c863-485f-4cfd-8d62-4b575e345798 tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:26:15.405673 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d4c863-485f-4cfd-8d62-4b575e345798 tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:15.407721 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d4c863-485f-4cfd-8d62-4b575e345798 tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:15.425989 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1018/2004] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:26:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:26:15.426968 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d4c863-485f-4cfd-8d62-4b575e345798 tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:15.444747 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 987/2005] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:26:15 2022] POST /identity/v3/users => generated 289 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:15.453070 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d4c863-485f-4cfd-8d62-4b575e345798 tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:15.459626 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a197c5b2-74f9-4dbe-a822-e2f89ab67091 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:15.461719 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f37b00-c2af-4a04-971f-6ccd75ef4c8c tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e741838ee19c4c719f2eea660a58119b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0f10f45a4e7452692cb2bab94adbe0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:15.462713 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f37b00-c2af-4a04-971f-6ccd75ef4c8c tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:15.463207 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f37b00-c2af-4a04-971f-6ccd75ef4c8c tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:15.463706 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f37b00-c2af-4a04-971f-6ccd75ef4c8c tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:15.464707 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f37b00-c2af-4a04-971f-6ccd75ef4c8c tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:15.467574 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f37b00-c2af-4a04-971f-6ccd75ef4c8c tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:15.493504 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a197c5b2-74f9-4dbe-a822-e2f89ab67091 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:15.507109 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:15.510316 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a219fcf-8ca5-4c33-a297-40dd12386607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:15.510971 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a219fcf-8ca5-4c33-a297-40dd12386607 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:15.511542 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a219fcf-8ca5-4c33-a297-40dd12386607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:15.511820 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a219fcf-8ca5-4c33-a297-40dd12386607 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:15.512651 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a219fcf-8ca5-4c33-a297-40dd12386607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:15.515617 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a219fcf-8ca5-4c33-a297-40dd12386607 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:15.550825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 988/2006] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:26:15.553598 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a219fcf-8ca5-4c33-a297-40dd12386607 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:15.578241 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a219fcf-8ca5-4c33-a297-40dd12386607 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:15.586484 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe3a38b-72ba-4ad2-9acc-29ba0bb6b500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:15.587348 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3a38b-72ba-4ad2-9acc-29ba0bb6b500 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:15.587928 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3a38b-72ba-4ad2-9acc-29ba0bb6b500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:15.588365 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3a38b-72ba-4ad2-9acc-29ba0bb6b500 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:15.589364 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe3a38b-72ba-4ad2-9acc-29ba0bb6b500 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:15.591520 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe3a38b-72ba-4ad2-9acc-29ba0bb6b500 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:15.598376 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1019/2007] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:26:15 2022] POST /identity/v3/projects => generated 353 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:15.603031 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f37b00-c2af-4a04-971f-6ccd75ef4c8c tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:15.609205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 989/2008] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:26:15.621632 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe3a38b-72ba-4ad2-9acc-29ba0bb6b500 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:15.639101 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe3a38b-72ba-4ad2-9acc-29ba0bb6b500 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:15.681893 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:15.751436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f37b00-c2af-4a04-971f-6ccd75ef4c8c tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:15.762051 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ec5e9-78e6-41d3-9aab-1e723ec9d472 tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e741838ee19c4c719f2eea660a58119b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0f10f45a4e7452692cb2bab94adbe0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:15.765817 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ec5e9-78e6-41d3-9aab-1e723ec9d472 tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:15.765817 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ec5e9-78e6-41d3-9aab-1e723ec9d472 tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:15.769655 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ec5e9-78e6-41d3-9aab-1e723ec9d472 tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:15.771068 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0ec5e9-78e6-41d3-9aab-1e723ec9d472 tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:26:15.771455 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0ec5e9-78e6-41d3-9aab-1e723ec9d472 tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:15.774457 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0ec5e9-78e6-41d3-9aab-1e723ec9d472 tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:15.779466 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211d7d7d-d378-4f39-bc87-fe18277fe6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:15.780158 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d7d7d-d378-4f39-bc87-fe18277fe6fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:15.780458 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d7d7d-d378-4f39-bc87-fe18277fe6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:15.780745 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d7d7d-d378-4f39-bc87-fe18277fe6fb admin admin] PATH_INFO: `/v3/projects/4d675dd935874f619876e2039ad5634b/users/e3cdc40e2a1842519d14046be74b3f78/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:15.792083 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1020/2009] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:26:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:26:15.793042 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ec5e9-78e6-41d3-9aab-1e723ec9d472 tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:15.794869 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-ce0ec5e9-78e6-41d3-9aab-1e723ec9d472 tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:15.807722 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211d7d7d-d378-4f39-bc87-fe18277fe6fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d675dd935874f619876e2039ad5634b, user_id=e3cdc40e2a1842519d14046be74b3f78, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:15.810528 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211d7d7d-d378-4f39-bc87-fe18277fe6fb admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:15.833680 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ec5e9-78e6-41d3-9aab-1e723ec9d472 tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:15.840082 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1597b62-5a7e-4bb6-962f-3cc1b90d96a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:15.840783 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1597b62-5a7e-4bb6-962f-3cc1b90d96a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:15.841064 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1597b62-5a7e-4bb6-962f-3cc1b90d96a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:15.841602 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1597b62-5a7e-4bb6-962f-3cc1b90d96a9 admin admin] PATH_INFO: `/v3/projects/3b45f8eb1f784c4f87d2d0560997f0ad` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:15.842901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1597b62-5a7e-4bb6-962f-3cc1b90d96a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b45f8eb1f784c4f87d2d0560997f0ad)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:15.844961 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1597b62-5a7e-4bb6-962f-3cc1b90d96a9 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:15.895484 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 990/2010] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:26:15 2022] PUT /identity/v3/projects/4d675dd935874f619876e2039ad5634b/users/e3cdc40e2a1842519d14046be74b3f78/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:15.895484 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211d7d7d-d378-4f39-bc87-fe18277fe6fb admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:15.965618 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211d7d7d-d378-4f39-bc87-fe18277fe6fb admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:15.971384 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504e7271-8dcd-49e0-837d-fc58f21f3967 tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e741838ee19c4c719f2eea660a58119b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0f10f45a4e7452692cb2bab94adbe0b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:15.973326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e7271-8dcd-49e0-837d-fc58f21f3967 tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:15.973633 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e7271-8dcd-49e0-837d-fc58f21f3967 tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:15.974027 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e7271-8dcd-49e0-837d-fc58f21f3967 tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] PATH_INFO: `/v3/projects/4b9cc8a8460c4c61b0faa371e24fbe09` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:15.976015 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504e7271-8dcd-49e0-837d-fc58f21f3967 tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] RBAC: Authorizing `identity:delete_project(project_id=4b9cc8a8460c4c61b0faa371e24fbe09)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:15.977980 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504e7271-8dcd-49e0-837d-fc58f21f3967 tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:16.001297 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-b1597b62-5a7e-4bb6-962f-3cc1b90d96a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b45f8eb1f784c4f87d2d0560997f0ad', 'request_id': 'req-b1597b62-5a7e-4bb6-962f-3cc1b90d96a9'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:16.011735 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-b1597b62-5a7e-4bb6-962f-3cc1b90d96a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b45f8eb1f784c4f87d2d0560997f0ad', 'request_id': 'req-b1597b62-5a7e-4bb6-962f-3cc1b90d96a9'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:16.019487 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-b1597b62-5a7e-4bb6-962f-3cc1b90d96a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b45f8eb1f784c4f87d2d0560997f0ad', 'request_id': 'req-b1597b62-5a7e-4bb6-962f-3cc1b90d96a9'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:16.171599 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-b1597b62-5a7e-4bb6-962f-3cc1b90d96a9 admin admin] The token cache is being invalidate because project 3b45f8eb1f784c4f87d2d0560997f0ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:26:16.321325 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-b1597b62-5a7e-4bb6-962f-3cc1b90d96a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:16.321325 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-b1597b62-5a7e-4bb6-962f-3cc1b90d96a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:16.321325 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1021/2011] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:26:15 2022] DELETE /identity/v3/projects/3b45f8eb1f784c4f87d2d0560997f0ad => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:16.321325 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1597b62-5a7e-4bb6-962f-3cc1b90d96a9 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:16.321325 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-b1597b62-5a7e-4bb6-962f-3cc1b90d96a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:16.321787 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-504e7271-8dcd-49e0-837d-fc58f21f3967 tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b9cc8a8460c4c61b0faa371e24fbe09', 'request_id': 'req-504e7271-8dcd-49e0-837d-fc58f21f3967'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:16.321787 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-504e7271-8dcd-49e0-837d-fc58f21f3967 tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b9cc8a8460c4c61b0faa371e24fbe09', 'request_id': 'req-504e7271-8dcd-49e0-837d-fc58f21f3967'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:16.322379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1597b62-5a7e-4bb6-962f-3cc1b90d96a9 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:16.322379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df343a81-8192-4ff9-8283-dde0c72b05f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:16.322665 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df343a81-8192-4ff9-8283-dde0c72b05f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:16.323002 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df343a81-8192-4ff9-8283-dde0c72b05f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:16.323284 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df343a81-8192-4ff9-8283-dde0c72b05f8 admin admin] PATH_INFO: `/v3/projects/6a671d549796458eb41071e4752c1b73` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:16.324597 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df343a81-8192-4ff9-8283-dde0c72b05f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a671d549796458eb41071e4752c1b73)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:16.326516 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df343a81-8192-4ff9-8283-dde0c72b05f8 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:16.470949 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-504e7271-8dcd-49e0-837d-fc58f21f3967 tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b9cc8a8460c4c61b0faa371e24fbe09', 'request_id': 'req-504e7271-8dcd-49e0-837d-fc58f21f3967'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:16.501149 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-504e7271-8dcd-49e0-837d-fc58f21f3967 tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] The token cache is being invalidate because project 4b9cc8a8460c4c61b0faa371e24fbe09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:26:16.568080 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-504e7271-8dcd-49e0-837d-fc58f21f3967 tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:16.568080 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-504e7271-8dcd-49e0-837d-fc58f21f3967 tempest-VolumeQuotaClassesTest-960161868 tempest-VolumeQuotaClassesTest-960161868-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:16.568080 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 991/2012] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:26:15 2022] DELETE /identity/v3/projects/4b9cc8a8460c4c61b0faa371e24fbe09 => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:16.568080 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0c88b-2a69-4e26-befb-c21bc8f6eef6 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:16.568080 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0c88b-2a69-4e26-befb-c21bc8f6eef6 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:16.569638 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0c88b-2a69-4e26-befb-c21bc8f6eef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:16.569638 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-62b0c88b-2a69-4e26-befb-c21bc8f6eef6 None None] MFA Rules not processed for user `e3cdc40e2a1842519d14046be74b3f78`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:16.569638 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-62b0c88b-2a69-4e26-befb-c21bc8f6eef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:16.641474 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 992/2013] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:16.673661 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b0c88b-2a69-4e26-befb-c21bc8f6eef6 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:16.673661 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-62b0c88b-2a69-4e26-befb-c21bc8f6eef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:16.673661 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b0c88b-2a69-4e26-befb-c21bc8f6eef6 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:16.682086 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1d35a-2a88-4d6d-a579-7d020597c853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:16.685330 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1d35a-2a88-4d6d-a579-7d020597c853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:16.685692 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1d35a-2a88-4d6d-a579-7d020597c853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:16.686052 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1d35a-2a88-4d6d-a579-7d020597c853 admin admin] PATH_INFO: `/v3/users/5530d1aca3c84774b95a0c7197cb01ff` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:16.687414 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c1d35a-2a88-4d6d-a579-7d020597c853 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5530d1aca3c84774b95a0c7197cb01ff)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:16.691507 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c1d35a-2a88-4d6d-a579-7d020597c853 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:16.739198 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-df343a81-8192-4ff9-8283-dde0c72b05f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a671d549796458eb41071e4752c1b73', 'request_id': 'req-df343a81-8192-4ff9-8283-dde0c72b05f8'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:16.747279 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-df343a81-8192-4ff9-8283-dde0c72b05f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a671d549796458eb41071e4752c1b73', 'request_id': 'req-df343a81-8192-4ff9-8283-dde0c72b05f8'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:16.754242 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-df343a81-8192-4ff9-8283-dde0c72b05f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a671d549796458eb41071e4752c1b73', 'request_id': 'req-df343a81-8192-4ff9-8283-dde0c72b05f8'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:16.884640 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-df343a81-8192-4ff9-8283-dde0c72b05f8 admin admin] The token cache is being invalidate because project 6a671d549796458eb41071e4752c1b73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:26:16.886937 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-df343a81-8192-4ff9-8283-dde0c72b05f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:16.886937 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-df343a81-8192-4ff9-8283-dde0c72b05f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:16.888777 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1022/2014] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:26:16 2022] DELETE /identity/v3/projects/6a671d549796458eb41071e4752c1b73 => generated 0 bytes in 711 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:16.890242 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df343a81-8192-4ff9-8283-dde0c72b05f8 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:16.893560 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-df343a81-8192-4ff9-8283-dde0c72b05f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:16.976227 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:17.000227 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-20c1d35a-2a88-4d6d-a579-7d020597c853 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5530d1aca3c84774b95a0c7197cb01ff'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:17.006224 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df343a81-8192-4ff9-8283-dde0c72b05f8 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:17.011899 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a95ca3c-a0af-466b-b2ea-1311d55428af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:17.012906 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95ca3c-a0af-466b-b2ea-1311d55428af admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:17.013879 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95ca3c-a0af-466b-b2ea-1311d55428af admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:17.014344 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95ca3c-a0af-466b-b2ea-1311d55428af admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:17.015281 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a95ca3c-a0af-466b-b2ea-1311d55428af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:17.017744 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a95ca3c-a0af-466b-b2ea-1311d55428af admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:17.042419 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-20c1d35a-2a88-4d6d-a579-7d020597c853 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5530d1aca3c84774b95a0c7197cb01ff'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:17.174819 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-20c1d35a-2a88-4d6d-a579-7d020597c853 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5530d1aca3c84774b95a0c7197cb01ff'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:17.177075 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-20c1d35a-2a88-4d6d-a579-7d020597c853 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5530d1aca3c84774b95a0c7197cb01ff'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:17.195873 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1023/2015] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:26:16 2022] POST /identity/v3/projects => generated 379 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:17.196065 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-20c1d35a-2a88-4d6d-a579-7d020597c853 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5530d1aca3c84774b95a0c7197cb01ff'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:17.197742 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a95ca3c-a0af-466b-b2ea-1311d55428af admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:17.203407 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 993/2016] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:26:16 2022] DELETE /identity/v3/users/5530d1aca3c84774b95a0c7197cb01ff => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:17.203407 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1d35a-2a88-4d6d-a579-7d020597c853 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:17.203687 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-20c1d35a-2a88-4d6d-a579-7d020597c853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:17.208197 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-1a95ca3c-a0af-466b-b2ea-1311d55428af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:17.296188 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a95ca3c-a0af-466b-b2ea-1311d55428af admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:17.333770 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1d35a-2a88-4d6d-a579-7d020597c853 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:17.346257 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea7a6fb-a157-4143-b5b6-1d708b73d518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:17.354437 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7a6fb-a157-4143-b5b6-1d708b73d518 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:17.354437 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7a6fb-a157-4143-b5b6-1d708b73d518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:17.354437 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7a6fb-a157-4143-b5b6-1d708b73d518 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:17.354437 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea7a6fb-a157-4143-b5b6-1d708b73d518 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:17.374622 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea7a6fb-a157-4143-b5b6-1d708b73d518 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:17.384630 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37e356-4ce8-4652-9fd6-6ee5167d2eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:17.385984 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37e356-4ce8-4652-9fd6-6ee5167d2eb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:17.386365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37e356-4ce8-4652-9fd6-6ee5167d2eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:17.389147 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37e356-4ce8-4652-9fd6-6ee5167d2eb6 admin admin] PATH_INFO: `/v3/users/4a7f0c09aeaf427085ecf514502afd65` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:17.390641 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb37e356-4ce8-4652-9fd6-6ee5167d2eb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a7f0c09aeaf427085ecf514502afd65)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:17.393686 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb37e356-4ce8-4652-9fd6-6ee5167d2eb6 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:17.488206 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 994/2017] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:26:17 2022] POST /identity/v3/users => generated 296 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:17.489462 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea7a6fb-a157-4143-b5b6-1d708b73d518 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:17.516619 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea7a6fb-a157-4143-b5b6-1d708b73d518 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:17.530174 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad625a9e-4d19-49c4-9c9a-a3e186010208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:17.530826 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad625a9e-4d19-49c4-9c9a-a3e186010208 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:17.531103 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad625a9e-4d19-49c4-9c9a-a3e186010208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:17.531368 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad625a9e-4d19-49c4-9c9a-a3e186010208 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:17.533222 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad625a9e-4d19-49c4-9c9a-a3e186010208 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:17.537570 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad625a9e-4d19-49c4-9c9a-a3e186010208 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:17.554628 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 995/2018] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:17 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:26:17.556216 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad625a9e-4d19-49c4-9c9a-a3e186010208 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:17.558123 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ad625a9e-4d19-49c4-9c9a-a3e186010208 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:17.578089 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad625a9e-4d19-49c4-9c9a-a3e186010208 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:17.584004 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e87e93-e772-48e5-a46e-9fc063bfad85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:17.584792 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e87e93-e772-48e5-a46e-9fc063bfad85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:17.585071 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e87e93-e772-48e5-a46e-9fc063bfad85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:17.585396 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e87e93-e772-48e5-a46e-9fc063bfad85 admin admin] PATH_INFO: `/v3/users/204af87dd73e45bda2e0cd4c98c92328` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:17.586651 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e87e93-e772-48e5-a46e-9fc063bfad85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=204af87dd73e45bda2e0cd4c98c92328)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:17.588600 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e87e93-e772-48e5-a46e-9fc063bfad85 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:17.638981 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-cb37e356-4ce8-4652-9fd6-6ee5167d2eb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a7f0c09aeaf427085ecf514502afd65'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:17.646846 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-cb37e356-4ce8-4652-9fd6-6ee5167d2eb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a7f0c09aeaf427085ecf514502afd65'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:17.647659 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-cb37e356-4ce8-4652-9fd6-6ee5167d2eb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a7f0c09aeaf427085ecf514502afd65'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:17.648263 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-cb37e356-4ce8-4652-9fd6-6ee5167d2eb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a7f0c09aeaf427085ecf514502afd65'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:17.663652 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-cb37e356-4ce8-4652-9fd6-6ee5167d2eb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a7f0c09aeaf427085ecf514502afd65'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:17.747790 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1024/2019] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:26:17 2022] DELETE /identity/v3/users/4a7f0c09aeaf427085ecf514502afd65 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:17.747790 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37e356-4ce8-4652-9fd6-6ee5167d2eb6 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:17.747790 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-cb37e356-4ce8-4652-9fd6-6ee5167d2eb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:17.805915 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-b6e87e93-e772-48e5-a46e-9fc063bfad85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '204af87dd73e45bda2e0cd4c98c92328'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:17.828282 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-b6e87e93-e772-48e5-a46e-9fc063bfad85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '204af87dd73e45bda2e0cd4c98c92328'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:17.839503 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=88,keystone:DELETE=6,keystone:INSERT=6 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:17.880885 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37e356-4ce8-4652-9fd6-6ee5167d2eb6 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:17.888031 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224c4f7-c760-4f31-9002-3df63f9c22d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:17.889316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224c4f7-c760-4f31-9002-3df63f9c22d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:17.889316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224c4f7-c760-4f31-9002-3df63f9c22d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:17.889591 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224c4f7-c760-4f31-9002-3df63f9c22d1 admin admin] PATH_INFO: `/v3/projects/697f0b95d9af4ddf952c83b1cd8caa44/users/3d65855b2cac425e845f57430efb204d/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:17.896901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-b6e87e93-e772-48e5-a46e-9fc063bfad85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '204af87dd73e45bda2e0cd4c98c92328'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:17.898584 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-b6e87e93-e772-48e5-a46e-9fc063bfad85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '204af87dd73e45bda2e0cd4c98c92328'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:17.910554 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-b6e87e93-e772-48e5-a46e-9fc063bfad85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '204af87dd73e45bda2e0cd4c98c92328'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:17.921831 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 996/2020] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:26:17 2022] DELETE /identity/v3/users/204af87dd73e45bda2e0cd4c98c92328 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:17.927536 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af792b-6602-461d-8099-f6cb78cf7abb None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:17.928241 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af792b-6602-461d-8099-f6cb78cf7abb None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:17.928757 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af792b-6602-461d-8099-f6cb78cf7abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:17.930164 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c224c4f7-c760-4f31-9002-3df63f9c22d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=697f0b95d9af4ddf952c83b1cd8caa44, user_id=3d65855b2cac425e845f57430efb204d, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:17.933817 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c224c4f7-c760-4f31-9002-3df63f9c22d1 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:17.961905 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-d9af792b-6602-461d-8099-f6cb78cf7abb None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:17.963997 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d9af792b-6602-461d-8099-f6cb78cf7abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:18.015207 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1025/2021] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:26:17 2022] PUT /identity/v3/projects/697f0b95d9af4ddf952c83b1cd8caa44/users/3d65855b2cac425e845f57430efb204d/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:18.019414 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224c4f7-c760-4f31-9002-3df63f9c22d1 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:18.023979 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-c224c4f7-c760-4f31-9002-3df63f9c22d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:18.080119 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 997/2022] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:18.122311 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224c4f7-c760-4f31-9002-3df63f9c22d1 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:18.125992 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af792b-6602-461d-8099-f6cb78cf7abb None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:18.127505 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d9af792b-6602-461d-8099-f6cb78cf7abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:18.129130 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb26779-b491-4e59-8d80-989cdc760bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:18.130019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb26779-b491-4e59-8d80-989cdc760bba admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:18.130455 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb26779-b491-4e59-8d80-989cdc760bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:18.130738 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb26779-b491-4e59-8d80-989cdc760bba admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:18.131460 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb26779-b491-4e59-8d80-989cdc760bba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:18.133187 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb26779-b491-4e59-8d80-989cdc760bba admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:18.169163 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1026/2023] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:18 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:26:18.173249 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb26779-b491-4e59-8d80-989cdc760bba admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:18.201742 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb26779-b491-4e59-8d80-989cdc760bba admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:18.213845 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2ba30-f87b-4659-b4e3-6528bbeec081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:18.214552 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2ba30-f87b-4659-b4e3-6528bbeec081 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:18.214832 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2ba30-f87b-4659-b4e3-6528bbeec081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:18.215114 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2ba30-f87b-4659-b4e3-6528bbeec081 admin admin] PATH_INFO: `/v3/projects/697f0b95d9af4ddf952c83b1cd8caa44/users/3d65855b2cac425e845f57430efb204d/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:18.226140 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe2ba30-f87b-4659-b4e3-6528bbeec081 admin admin] RBAC: Authorizing `identity:create_grant(project_id=697f0b95d9af4ddf952c83b1cd8caa44, user_id=3d65855b2cac425e845f57430efb204d, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:18.228475 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe2ba30-f87b-4659-b4e3-6528bbeec081 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:18.250842 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1027/2024] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:26:18 2022] PUT /identity/v3/projects/697f0b95d9af4ddf952c83b1cd8caa44/users/3d65855b2cac425e845f57430efb204d/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:18.256806 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2ba30-f87b-4659-b4e3-6528bbeec081 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:18.259249 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-1fe2ba30-f87b-4659-b4e3-6528bbeec081 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:18.306112 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af792b-6602-461d-8099-f6cb78cf7abb None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:18.399882 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f41ce2-ef9b-46d1-859a-3ff9a511d0a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:18.400471 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f41ce2-ef9b-46d1-859a-3ff9a511d0a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:18.400752 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f41ce2-ef9b-46d1-859a-3ff9a511d0a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:18.401025 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f41ce2-ef9b-46d1-859a-3ff9a511d0a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:18.406326 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2ba30-f87b-4659-b4e3-6528bbeec081 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:18.413690 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d83cac-c8eb-412a-bbc3-dbeafd1aa373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:18.414574 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d83cac-c8eb-412a-bbc3-dbeafd1aa373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:18.415325 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d83cac-c8eb-412a-bbc3-dbeafd1aa373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:18.415325 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d83cac-c8eb-412a-bbc3-dbeafd1aa373 admin admin] PATH_INFO: `/v3/projects/27d0699cdab14f7a8fad4c29c40978e4` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:18.416911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d83cac-c8eb-412a-bbc3-dbeafd1aa373 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27d0699cdab14f7a8fad4c29c40978e4)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:18.418988 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d83cac-c8eb-412a-bbc3-dbeafd1aa373 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:18.420980 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f41ce2-ef9b-46d1-859a-3ff9a511d0a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:18.423176 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f41ce2-ef9b-46d1-859a-3ff9a511d0a7 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:18.452435 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 998/2025] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:26:18.454186 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222fdcf-7414-4ef9-9b98-5fa9618fa24f None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:18.454477 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222fdcf-7414-4ef9-9b98-5fa9618fa24f None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:18.454751 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222fdcf-7414-4ef9-9b98-5fa9618fa24f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:18.491694 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-1222fdcf-7414-4ef9-9b98-5fa9618fa24f None None] MFA Rules not processed for user `3d65855b2cac425e845f57430efb204d`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:18.492865 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-1222fdcf-7414-4ef9-9b98-5fa9618fa24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:18.577785 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:18.581588 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-62d83cac-c8eb-412a-bbc3-dbeafd1aa373 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27d0699cdab14f7a8fad4c29c40978e4', 'request_id': 'req-62d83cac-c8eb-412a-bbc3-dbeafd1aa373'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:18.610174 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-62d83cac-c8eb-412a-bbc3-dbeafd1aa373 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27d0699cdab14f7a8fad4c29c40978e4', 'request_id': 'req-62d83cac-c8eb-412a-bbc3-dbeafd1aa373'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:18.622464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-62d83cac-c8eb-412a-bbc3-dbeafd1aa373 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27d0699cdab14f7a8fad4c29c40978e4', 'request_id': 'req-62d83cac-c8eb-412a-bbc3-dbeafd1aa373'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:18.653735 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 999/2026] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:18 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:18.665600 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222fdcf-7414-4ef9-9b98-5fa9618fa24f None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:18.667438 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-1222fdcf-7414-4ef9-9b98-5fa9618fa24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:18.714952 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-62d83cac-c8eb-412a-bbc3-dbeafd1aa373 admin admin] The token cache is being invalidate because project 27d0699cdab14f7a8fad4c29c40978e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:26:18.715321 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-62d83cac-c8eb-412a-bbc3-dbeafd1aa373 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:18.717133 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-62d83cac-c8eb-412a-bbc3-dbeafd1aa373 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:18.723463 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1028/2027] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:26:18 2022] DELETE /identity/v3/projects/27d0699cdab14f7a8fad4c29c40978e4 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:26:18.725088 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb6017-2846-4838-97b8-8153ae1caee9 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:18.725380 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb6017-2846-4838-97b8-8153ae1caee9 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:18.725655 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb6017-2846-4838-97b8-8153ae1caee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:18.765613 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222fdcf-7414-4ef9-9b98-5fa9618fa24f None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:18.768715 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-afdb6017-2846-4838-97b8-8153ae1caee9 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:18.770353 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-afdb6017-2846-4838-97b8-8153ae1caee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:18.832878 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07503f27-99ce-4ec4-9c96-339bb8c1572d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:18.833524 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07503f27-99ce-4ec4-9c96-339bb8c1572d service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:18.833806 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07503f27-99ce-4ec4-9c96-339bb8c1572d service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:18.834078 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07503f27-99ce-4ec4-9c96-339bb8c1572d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:18.836074 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-07503f27-99ce-4ec4-9c96-339bb8c1572d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:18.873584 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1029/2028] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:18.876114 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4a35a-19de-47fc-9eae-031574a60317 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:18.876445 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4a35a-19de-47fc-9eae-031574a60317 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:18.876764 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4a35a-19de-47fc-9eae-031574a60317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:18.914376 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-4aa4a35a-19de-47fc-9eae-031574a60317 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:18.916074 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-4aa4a35a-19de-47fc-9eae-031574a60317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:18.931798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1030/2029] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:26:18.934696 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa4a35a-19de-47fc-9eae-031574a60317 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:18.946903 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa4a35a-19de-47fc-9eae-031574a60317 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:18.953404 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03c698-6925-41a8-b57e-08f430c5e24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:18.954147 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03c698-6925-41a8-b57e-08f430c5e24e admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:18.954425 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03c698-6925-41a8-b57e-08f430c5e24e admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:18.954838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03c698-6925-41a8-b57e-08f430c5e24e admin admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:18.955624 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d03c698-6925-41a8-b57e-08f430c5e24e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:26:18.956718 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d03c698-6925-41a8-b57e-08f430c5e24e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:18.958612 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d03c698-6925-41a8-b57e-08f430c5e24e admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:18.973024 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07503f27-99ce-4ec4-9c96-339bb8c1572d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:18.974549 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07503f27-99ce-4ec4-9c96-339bb8c1572d service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:18.978306 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1031/2030] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:26:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:26:18.979297 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03c698-6925-41a8-b57e-08f430c5e24e admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:18.989387 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1000/2031] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:18 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:26:18.990659 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03c698-6925-41a8-b57e-08f430c5e24e admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:19.003740 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81884725-a78b-4870-8041-6fc303b556c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:19.004829 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81884725-a78b-4870-8041-6fc303b556c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:19.005306 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81884725-a78b-4870-8041-6fc303b556c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:19.005739 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81884725-a78b-4870-8041-6fc303b556c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:19.006569 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81884725-a78b-4870-8041-6fc303b556c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:19.008851 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81884725-a78b-4870-8041-6fc303b556c5 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:19.019504 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07503f27-99ce-4ec4-9c96-339bb8c1572d service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:19.021218 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-07503f27-99ce-4ec4-9c96-339bb8c1572d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:19.049010 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07503f27-99ce-4ec4-9c96-339bb8c1572d service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:19.058243 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b341a-346f-4cc7-b2d3-5ad6b67fb469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:19.058910 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b341a-346f-4cc7-b2d3-5ad6b67fb469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:19.059190 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b341a-346f-4cc7-b2d3-5ad6b67fb469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:19.059467 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b341a-346f-4cc7-b2d3-5ad6b67fb469 admin admin] PATH_INFO: `/v3/projects/990173ac30f54cf5876b635ff3cfb184` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:19.061260 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81b341a-346f-4cc7-b2d3-5ad6b67fb469 admin admin] RBAC: Authorizing `identity:delete_project(project_id=990173ac30f54cf5876b635ff3cfb184)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:19.063435 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81b341a-346f-4cc7-b2d3-5ad6b67fb469 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:19.180226 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1032/2032] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:26:18 2022] POST /identity/v3/projects => generated 361 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:19.181673 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81884725-a78b-4870-8041-6fc303b556c5 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:19.209205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c81b341a-346f-4cc7-b2d3-5ad6b67fb469 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '990173ac30f54cf5876b635ff3cfb184', 'request_id': 'req-c81b341a-346f-4cc7-b2d3-5ad6b67fb469'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:19.216801 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c81b341a-346f-4cc7-b2d3-5ad6b67fb469 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '990173ac30f54cf5876b635ff3cfb184', 'request_id': 'req-c81b341a-346f-4cc7-b2d3-5ad6b67fb469'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:19.268126 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81884725-a78b-4870-8041-6fc303b556c5 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:19.302899 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c81b341a-346f-4cc7-b2d3-5ad6b67fb469 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '990173ac30f54cf5876b635ff3cfb184', 'request_id': 'req-c81b341a-346f-4cc7-b2d3-5ad6b67fb469'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:19.321397 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c81b341a-346f-4cc7-b2d3-5ad6b67fb469 admin admin] The token cache is being invalidate because project 990173ac30f54cf5876b635ff3cfb184 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:26:19.321724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c81b341a-346f-4cc7-b2d3-5ad6b67fb469 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:19.322901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c81b341a-346f-4cc7-b2d3-5ad6b67fb469 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:19.325250 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1001/2033] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:26:19 2022] DELETE /identity/v3/projects/990173ac30f54cf5876b635ff3cfb184 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:26:19.327094 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b341a-346f-4cc7-b2d3-5ad6b67fb469 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:19.334978 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c81b341a-346f-4cc7-b2d3-5ad6b67fb469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:19.337798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52666d9-c31c-4e1d-b803-0793f287d4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:19.337798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52666d9-c31c-4e1d-b803-0793f287d4b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:19.337798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52666d9-c31c-4e1d-b803-0793f287d4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:19.337798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52666d9-c31c-4e1d-b803-0793f287d4b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:19.338391 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52666d9-c31c-4e1d-b803-0793f287d4b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:19.341319 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52666d9-c31c-4e1d-b803-0793f287d4b4 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:19.353810 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b341a-346f-4cc7-b2d3-5ad6b67fb469 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:19.361818 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc62cb-96e7-4f9a-be93-810a84b357db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:19.362491 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc62cb-96e7-4f9a-be93-810a84b357db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:19.362854 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc62cb-96e7-4f9a-be93-810a84b357db admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:19.364075 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc62cb-96e7-4f9a-be93-810a84b357db admin admin] PATH_INFO: `/v3/projects/4befd2689cb84279bc430d344aba0ec4` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:19.365418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdc62cb-96e7-4f9a-be93-810a84b357db admin admin] RBAC: Authorizing `identity:delete_project(project_id=4befd2689cb84279bc430d344aba0ec4)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:19.369150 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdc62cb-96e7-4f9a-be93-810a84b357db admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:19.436453 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1033/2034] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:26:19 2022] POST /identity/v3/users => generated 281 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:19.436453 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52666d9-c31c-4e1d-b803-0793f287d4b4 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:19.436897 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-a52666d9-c31c-4e1d-b803-0793f287d4b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:19.492731 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-afdc62cb-96e7-4f9a-be93-810a84b357db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4befd2689cb84279bc430d344aba0ec4', 'request_id': 'req-afdc62cb-96e7-4f9a-be93-810a84b357db'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:19.496676 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52666d9-c31c-4e1d-b803-0793f287d4b4 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:19.499797 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-afdc62cb-96e7-4f9a-be93-810a84b357db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4befd2689cb84279bc430d344aba0ec4', 'request_id': 'req-afdc62cb-96e7-4f9a-be93-810a84b357db'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:19.551158 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a20d04b-f765-48e7-908d-c64159f195e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:19.551782 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20d04b-f765-48e7-908d-c64159f195e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:19.552053 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20d04b-f765-48e7-908d-c64159f195e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:19.552325 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20d04b-f765-48e7-908d-c64159f195e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:19.554210 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-0a20d04b-f765-48e7-908d-c64159f195e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:19.583384 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-afdc62cb-96e7-4f9a-be93-810a84b357db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4befd2689cb84279bc430d344aba0ec4', 'request_id': 'req-afdc62cb-96e7-4f9a-be93-810a84b357db'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:19.595826 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-afdc62cb-96e7-4f9a-be93-810a84b357db admin admin] The token cache is being invalidate because project 4befd2689cb84279bc430d344aba0ec4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:26:19.599269 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-afdc62cb-96e7-4f9a-be93-810a84b357db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:19.599269 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-afdc62cb-96e7-4f9a-be93-810a84b357db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:19.599269 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1002/2035] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:26:19 2022] DELETE /identity/v3/projects/4befd2689cb84279bc430d344aba0ec4 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:19.600206 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bfe8c0-cf9d-4b1a-b8c6-757ed4c5087f None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:19.600522 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bfe8c0-cf9d-4b1a-b8c6-757ed4c5087f None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:19.600792 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bfe8c0-cf9d-4b1a-b8c6-757ed4c5087f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:19.624052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a20d04b-f765-48e7-908d-c64159f195e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:19.625586 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a20d04b-f765-48e7-908d-c64159f195e6 service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:19.626555 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-e4bfe8c0-cf9d-4b1a-b8c6-757ed4c5087f None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:19.629065 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e4bfe8c0-cf9d-4b1a-b8c6-757ed4c5087f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:19.635994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1034/2036] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:19 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:26:19.636798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a20d04b-f765-48e7-908d-c64159f195e6 service cinder] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:19.638202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-0a20d04b-f765-48e7-908d-c64159f195e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:19.653150 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:19.729212 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a20d04b-f765-48e7-908d-c64159f195e6 service cinder] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:19.735003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681ea7a-8bd7-4b11-861a-e8e5788f7823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:19.735767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681ea7a-8bd7-4b11-861a-e8e5788f7823 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:19.736054 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681ea7a-8bd7-4b11-861a-e8e5788f7823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:19.736331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681ea7a-8bd7-4b11-861a-e8e5788f7823 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:19.737818 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1681ea7a-8bd7-4b11-861a-e8e5788f7823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:19.739016 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1003/2037] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:19.739719 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1681ea7a-8bd7-4b11-861a-e8e5788f7823 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:19.740696 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bfe8c0-cf9d-4b1a-b8c6-757ed4c5087f None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:19.743448 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e4bfe8c0-cf9d-4b1a-b8c6-757ed4c5087f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:19.751852 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1035/2038] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:19 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:26:19.753890 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48eb0c9-5d9b-4779-a42f-7b6b66d7f5b9 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:19.754234 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48eb0c9-5d9b-4779-a42f-7b6b66d7f5b9 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:19.754516 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48eb0c9-5d9b-4779-a42f-7b6b66d7f5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:19.767707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bfe8c0-cf9d-4b1a-b8c6-757ed4c5087f None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:19.770522 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b1f6cc-ba85-46dc-9a84-63e051dbf0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:19.771779 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1f6cc-ba85-46dc-9a84-63e051dbf0aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:19.773215 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1f6cc-ba85-46dc-9a84-63e051dbf0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:19.773683 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1f6cc-ba85-46dc-9a84-63e051dbf0aa admin admin] PATH_INFO: `/v3/users/e741838ee19c4c719f2eea660a58119b` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:19.775210 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b1f6cc-ba85-46dc-9a84-63e051dbf0aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=e741838ee19c4c719f2eea660a58119b)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:19.777389 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b1f6cc-ba85-46dc-9a84-63e051dbf0aa admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:19.789597 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-b48eb0c9-5d9b-4779-a42f-7b6b66d7f5b9 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:19.791570 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-b48eb0c9-5d9b-4779-a42f-7b6b66d7f5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:19.803623 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1036/2039] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:19.804497 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48eb0c9-5d9b-4779-a42f-7b6b66d7f5b9 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:19.806001 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-b48eb0c9-5d9b-4779-a42f-7b6b66d7f5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:19.824092 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48eb0c9-5d9b-4779-a42f-7b6b66d7f5b9 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:19.829526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58a499f-364b-40ee-a559-aeb2599079df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:19.835059 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58a499f-364b-40ee-a559-aeb2599079df admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:19.835059 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58a499f-364b-40ee-a559-aeb2599079df admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:19.835059 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58a499f-364b-40ee-a559-aeb2599079df admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:19.835059 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58a499f-364b-40ee-a559-aeb2599079df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:19.835059 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58a499f-364b-40ee-a559-aeb2599079df admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:19.852268 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1037/2040] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:19 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:26:19.852268 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58a499f-364b-40ee-a559-aeb2599079df admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:19.865258 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58a499f-364b-40ee-a559-aeb2599079df admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:19.868473 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc10b84-f239-473b-903f-9f4f93abe3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:19.869117 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc10b84-f239-473b-903f-9f4f93abe3fa admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:19.869395 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc10b84-f239-473b-903f-9f4f93abe3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:19.869667 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc10b84-f239-473b-903f-9f4f93abe3fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:19.871314 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc10b84-f239-473b-903f-9f4f93abe3fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:26:19.871314 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc10b84-f239-473b-903f-9f4f93abe3fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:19.872616 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc10b84-f239-473b-903f-9f4f93abe3fa admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:19.889238 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1038/2041] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:26:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:26:19.890392 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc10b84-f239-473b-903f-9f4f93abe3fa admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:19.911810 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc10b84-f239-473b-903f-9f4f93abe3fa admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:19.918368 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e201ad4-0570-41b4-b85f-00fc24b8357c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:19.919305 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e201ad4-0570-41b4-b85f-00fc24b8357c admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:19.919660 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e201ad4-0570-41b4-b85f-00fc24b8357c admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:19.920002 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e201ad4-0570-41b4-b85f-00fc24b8357c admin admin] PATH_INFO: `/v3/projects/68e5168b1f42490fb94aecf8e0659d54/users/8062e8c8fb1444999faf2b55e397028a/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:19.953632 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e201ad4-0570-41b4-b85f-00fc24b8357c admin admin] RBAC: Authorizing `identity:create_grant(project_id=68e5168b1f42490fb94aecf8e0659d54, user_id=8062e8c8fb1444999faf2b55e397028a, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:19.958048 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e201ad4-0570-41b4-b85f-00fc24b8357c admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:20.082308 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a4b1f6cc-ba85-46dc-9a84-63e051dbf0aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e741838ee19c4c719f2eea660a58119b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:20.094238 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a4b1f6cc-ba85-46dc-9a84-63e051dbf0aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e741838ee19c4c719f2eea660a58119b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:20.109043 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1039/2042] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:26:19 2022] PUT /identity/v3/projects/68e5168b1f42490fb94aecf8e0659d54/users/8062e8c8fb1444999faf2b55e397028a/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:20.109930 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e201ad4-0570-41b4-b85f-00fc24b8357c admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:20.172846 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e201ad4-0570-41b4-b85f-00fc24b8357c admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:20.177681 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7ef5fd-fee3-4a05-8a9a-00a003d62eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:20.178299 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ef5fd-fee3-4a05-8a9a-00a003d62eed admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:20.178582 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ef5fd-fee3-4a05-8a9a-00a003d62eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:20.178857 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ef5fd-fee3-4a05-8a9a-00a003d62eed admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:20.179522 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7ef5fd-fee3-4a05-8a9a-00a003d62eed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:20.181541 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7ef5fd-fee3-4a05-8a9a-00a003d62eed admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:20.205294 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a4b1f6cc-ba85-46dc-9a84-63e051dbf0aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e741838ee19c4c719f2eea660a58119b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:20.215672 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a4b1f6cc-ba85-46dc-9a84-63e051dbf0aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e741838ee19c4c719f2eea660a58119b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:20.216513 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a4b1f6cc-ba85-46dc-9a84-63e051dbf0aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e741838ee19c4c719f2eea660a58119b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:20.223809 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1004/2043] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:26:19 2022] DELETE /identity/v3/users/e741838ee19c4c719f2eea660a58119b => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:20.225330 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5750f3f9-564b-4ac9-bcdb-18d0e3dad769 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:20.225726 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5750f3f9-564b-4ac9-bcdb-18d0e3dad769 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:20.226144 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5750f3f9-564b-4ac9-bcdb-18d0e3dad769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:20.253768 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-5750f3f9-564b-4ac9-bcdb-18d0e3dad769 None None] MFA Rules not processed for user `8062e8c8fb1444999faf2b55e397028a`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:20.255086 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5750f3f9-564b-4ac9-bcdb-18d0e3dad769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:20.294047 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1040/2044] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:26:20 2022] POST /identity/v3/projects => generated 385 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:20.306668 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7ef5fd-fee3-4a05-8a9a-00a003d62eed admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:20.306668 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-0d7ef5fd-fee3-4a05-8a9a-00a003d62eed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:20.322719 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1005/2045] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:20.359660 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5750f3f9-564b-4ac9-bcdb-18d0e3dad769 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:20.361154 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5750f3f9-564b-4ac9-bcdb-18d0e3dad769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:20.376977 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7ef5fd-fee3-4a05-8a9a-00a003d62eed admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:20.382247 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96817a44-a3c0-4002-adb6-0d8f4db246da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:20.382862 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96817a44-a3c0-4002-adb6-0d8f4db246da admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:20.383140 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96817a44-a3c0-4002-adb6-0d8f4db246da admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:20.383412 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96817a44-a3c0-4002-adb6-0d8f4db246da admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:20.384071 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96817a44-a3c0-4002-adb6-0d8f4db246da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:20.386021 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96817a44-a3c0-4002-adb6-0d8f4db246da admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:20.419340 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5750f3f9-564b-4ac9-bcdb-18d0e3dad769 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:20.475861 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0486f9c2-f3df-41db-8363-c4f07411b4f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:20.476591 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486f9c2-f3df-41db-8363-c4f07411b4f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:20.476874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486f9c2-f3df-41db-8363-c4f07411b4f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:20.477151 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486f9c2-f3df-41db-8363-c4f07411b4f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:20.514785 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1041/2046] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:26:20 2022] POST /identity/v3/users => generated 293 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:26:20.518863 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96817a44-a3c0-4002-adb6-0d8f4db246da admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:20.530390 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96817a44-a3c0-4002-adb6-0d8f4db246da admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:20.537245 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f9833-7667-4abe-af4a-9b88fa975dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:20.538158 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f9833-7667-4abe-af4a-9b88fa975dd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:20.538493 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f9833-7667-4abe-af4a-9b88fa975dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:20.538963 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f9833-7667-4abe-af4a-9b88fa975dd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:20.539687 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3f9833-7667-4abe-af4a-9b88fa975dd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:20.541465 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3f9833-7667-4abe-af4a-9b88fa975dd3 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:20.546649 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0486f9c2-f3df-41db-8363-c4f07411b4f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:20.548105 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0486f9c2-f3df-41db-8363-c4f07411b4f6 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:20.553958 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1042/2047] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:26:20.562828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1006/2048] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 14:26:20.566838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f9833-7667-4abe-af4a-9b88fa975dd3 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:20.576083 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0486f9c2-f3df-41db-8363-c4f07411b4f6 service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:20.577785 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-0486f9c2-f3df-41db-8363-c4f07411b4f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:20.579904 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f9833-7667-4abe-af4a-9b88fa975dd3 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:20.585653 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c2fad-43f5-4721-b620-c4906a68961d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:20.586437 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c2fad-43f5-4721-b620-c4906a68961d admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:20.586875 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c2fad-43f5-4721-b620-c4906a68961d admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:20.587328 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c2fad-43f5-4721-b620-c4906a68961d admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:20.588199 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278c2fad-43f5-4721-b620-c4906a68961d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:20.590026 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278c2fad-43f5-4721-b620-c4906a68961d admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:20.597548 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0486f9c2-f3df-41db-8363-c4f07411b4f6 service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:20.601532 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1043/2049] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:26:20.603393 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e96b44-9b7c-45a0-b021-750a5bca8109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:20.604037 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e96b44-9b7c-45a0-b021-750a5bca8109 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:20.604317 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e96b44-9b7c-45a0-b021-750a5bca8109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:20.604624 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e96b44-9b7c-45a0-b021-750a5bca8109 admin admin] PATH_INFO: `/v3/users/8062e8c8fb1444999faf2b55e397028a` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:20.605812 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e96b44-9b7c-45a0-b021-750a5bca8109 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8062e8c8fb1444999faf2b55e397028a)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:20.606938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c2fad-43f5-4721-b620-c4906a68961d admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:20.607769 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e96b44-9b7c-45a0-b021-750a5bca8109 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:20.620187 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c2fad-43f5-4721-b620-c4906a68961d admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:20.625002 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10223f5d-9ab4-4615-803d-d456233b5e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:20.625719 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10223f5d-9ab4-4615-803d-d456233b5e96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:20.626040 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10223f5d-9ab4-4615-803d-d456233b5e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:20.626265 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10223f5d-9ab4-4615-803d-d456233b5e96 admin admin] PATH_INFO: `/v3/projects/ade6d833426847fcb9bf2ee7227f4c63/users/9c11b7ea92f44d5f98dacd3de11ebacf/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:20.642533 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10223f5d-9ab4-4615-803d-d456233b5e96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ade6d833426847fcb9bf2ee7227f4c63, user_id=9c11b7ea92f44d5f98dacd3de11ebacf, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:20.646188 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10223f5d-9ab4-4615-803d-d456233b5e96 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:20.702886 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:20.765627 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1044/2050] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:26:20 2022] PUT /identity/v3/projects/ade6d833426847fcb9bf2ee7227f4c63/users/9c11b7ea92f44d5f98dacd3de11ebacf/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:20.769120 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10223f5d-9ab4-4615-803d-d456233b5e96 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:20.769120 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-10223f5d-9ab4-4615-803d-d456233b5e96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:20.804466 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-37e96b44-9b7c-45a0-b021-750a5bca8109 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8062e8c8fb1444999faf2b55e397028a'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:20.822094 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-37e96b44-9b7c-45a0-b021-750a5bca8109 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8062e8c8fb1444999faf2b55e397028a'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:20.838204 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10223f5d-9ab4-4615-803d-d456233b5e96 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:20.843327 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310489a2-da0b-4d9a-a09f-c8dfb6963295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:20.843972 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310489a2-da0b-4d9a-a09f-c8dfb6963295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:20.844252 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310489a2-da0b-4d9a-a09f-c8dfb6963295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:20.844559 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310489a2-da0b-4d9a-a09f-c8dfb6963295 admin admin] PATH_INFO: `/v3/projects/c0f10f45a4e7452692cb2bab94adbe0b` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:20.845749 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310489a2-da0b-4d9a-a09f-c8dfb6963295 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0f10f45a4e7452692cb2bab94adbe0b)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:20.847896 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310489a2-da0b-4d9a-a09f-c8dfb6963295 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:20.906562 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-37e96b44-9b7c-45a0-b021-750a5bca8109 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8062e8c8fb1444999faf2b55e397028a'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:20.907203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-37e96b44-9b7c-45a0-b021-750a5bca8109 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8062e8c8fb1444999faf2b55e397028a'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:20.915299 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-37e96b44-9b7c-45a0-b021-750a5bca8109 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8062e8c8fb1444999faf2b55e397028a'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:20.921196 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1007/2051] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:26:20 2022] DELETE /identity/v3/users/8062e8c8fb1444999faf2b55e397028a => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:26:20.922400 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f40fe-fd6c-4294-8a7f-a90e5499c9e4 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:20.922679 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f40fe-fd6c-4294-8a7f-a90e5499c9e4 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:20.922946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f40fe-fd6c-4294-8a7f-a90e5499c9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:20.949934 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-126f40fe-fd6c-4294-8a7f-a90e5499c9e4 None None] MFA Rules not processed for user `9c11b7ea92f44d5f98dacd3de11ebacf`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:20.951137 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-126f40fe-fd6c-4294-8a7f-a90e5499c9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:21.011721 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-310489a2-da0b-4d9a-a09f-c8dfb6963295 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0f10f45a4e7452692cb2bab94adbe0b', 'request_id': 'req-310489a2-da0b-4d9a-a09f-c8dfb6963295'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:21.014446 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1008/2052] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:21.015694 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb7c50-c238-49b4-808b-05e2fc1fded5 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:21.016102 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb7c50-c238-49b4-808b-05e2fc1fded5 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:21.016394 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb7c50-c238-49b4-808b-05e2fc1fded5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:21.025479 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-310489a2-da0b-4d9a-a09f-c8dfb6963295 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0f10f45a4e7452692cb2bab94adbe0b', 'request_id': 'req-310489a2-da0b-4d9a-a09f-c8dfb6963295'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:21.031352 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-310489a2-da0b-4d9a-a09f-c8dfb6963295 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0f10f45a4e7452692cb2bab94adbe0b', 'request_id': 'req-310489a2-da0b-4d9a-a09f-c8dfb6963295'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:21.036168 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-d1bb7c50-c238-49b4-808b-05e2fc1fded5 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:21.037436 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d1bb7c50-c238-49b4-808b-05e2fc1fded5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:21.097650 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1009/2053] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:21.098618 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb7c50-c238-49b4-808b-05e2fc1fded5 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:21.100036 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d1bb7c50-c238-49b4-808b-05e2fc1fded5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:21.120266 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-310489a2-da0b-4d9a-a09f-c8dfb6963295 admin admin] The token cache is being invalidate because project c0f10f45a4e7452692cb2bab94adbe0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:26:21.121106 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-310489a2-da0b-4d9a-a09f-c8dfb6963295 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:21.123139 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-310489a2-da0b-4d9a-a09f-c8dfb6963295 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:21.124963 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1045/2054] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:26:20 2022] DELETE /identity/v3/projects/c0f10f45a4e7452692cb2bab94adbe0b => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:21.125951 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310489a2-da0b-4d9a-a09f-c8dfb6963295 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:21.127553 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-310489a2-da0b-4d9a-a09f-c8dfb6963295 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:21.161375 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb7c50-c238-49b4-808b-05e2fc1fded5 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:21.165793 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db5397-fd7e-42ed-a4be-8ee9c3d65f1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:21.166609 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db5397-fd7e-42ed-a4be-8ee9c3d65f1e service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:21.166952 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db5397-fd7e-42ed-a4be-8ee9c3d65f1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:21.167228 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db5397-fd7e-42ed-a4be-8ee9c3d65f1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:21.170627 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-05db5397-fd7e-42ed-a4be-8ee9c3d65f1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:21.191859 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310489a2-da0b-4d9a-a09f-c8dfb6963295 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:21.195867 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3de644-fb9a-4036-8314-9cdc3cce2beb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:21.196531 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3de644-fb9a-4036-8314-9cdc3cce2beb service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:21.196815 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3de644-fb9a-4036-8314-9cdc3cce2beb service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:21.197460 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3de644-fb9a-4036-8314-9cdc3cce2beb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:21.200533 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-2b3de644-fb9a-4036-8314-9cdc3cce2beb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:21.220237 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3de644-fb9a-4036-8314-9cdc3cce2beb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:21.224467 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3de644-fb9a-4036-8314-9cdc3cce2beb service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:21.235280 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05db5397-fd7e-42ed-a4be-8ee9c3d65f1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:21.237164 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05db5397-fd7e-42ed-a4be-8ee9c3d65f1e service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:21.241416 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1046/2055] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:26:21.242907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfbd435-17ca-43c6-a434-f78c95e80b57 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:21.243187 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfbd435-17ca-43c6-a434-f78c95e80b57 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:21.243470 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfbd435-17ca-43c6-a434-f78c95e80b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:21.247231 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1010/2056] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:26:21.266851 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-dcfbd435-17ca-43c6-a434-f78c95e80b57 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:21.268116 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-dcfbd435-17ca-43c6-a434-f78c95e80b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:21.277373 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1047/2057] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:21.285385 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b82c7-e189-4323-8fb0-6180ec87bf0e None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:21.285674 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b82c7-e189-4323-8fb0-6180ec87bf0e None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:21.285970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b82c7-e189-4323-8fb0-6180ec87bf0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:21.298194 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:21.318470 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-d36b82c7-e189-4323-8fb0-6180ec87bf0e None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:21.320160 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d36b82c7-e189-4323-8fb0-6180ec87bf0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:21.332303 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1011/2058] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:21.334048 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfbd435-17ca-43c6-a434-f78c95e80b57 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:21.345274 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfbd435-17ca-43c6-a434-f78c95e80b57 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:21.350451 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8736a1fa-d3d2-4f41-b8b6-d5f832e2ac38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:21.351148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736a1fa-d3d2-4f41-b8b6-d5f832e2ac38 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:21.351500 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736a1fa-d3d2-4f41-b8b6-d5f832e2ac38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:21.351784 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736a1fa-d3d2-4f41-b8b6-d5f832e2ac38 admin admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:21.352502 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8736a1fa-d3d2-4f41-b8b6-d5f832e2ac38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:26:21.352844 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8736a1fa-d3d2-4f41-b8b6-d5f832e2ac38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:21.354453 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8736a1fa-d3d2-4f41-b8b6-d5f832e2ac38 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:21.366472 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1048/2059] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:26:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:26:21.371046 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b82c7-e189-4323-8fb0-6180ec87bf0e None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:21.381537 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b82c7-e189-4323-8fb0-6180ec87bf0e None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:21.386405 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a79a342-5968-4a16-8e33-8eb7293f9b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:21.389408 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79a342-5968-4a16-8e33-8eb7293f9b99 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:21.389408 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79a342-5968-4a16-8e33-8eb7293f9b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:21.389408 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79a342-5968-4a16-8e33-8eb7293f9b99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:21.389408 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a79a342-5968-4a16-8e33-8eb7293f9b99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:21.390324 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a79a342-5968-4a16-8e33-8eb7293f9b99 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:21.489575 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1012/2060] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:26:21 2022] POST /identity/v3/projects => generated 365 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:21.493583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8736a1fa-d3d2-4f41-b8b6-d5f832e2ac38 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:21.505259 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8736a1fa-d3d2-4f41-b8b6-d5f832e2ac38 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:21.563818 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42f56c-347a-4124-af11-8e1198e09131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:21.564462 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42f56c-347a-4124-af11-8e1198e09131 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:21.564871 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42f56c-347a-4124-af11-8e1198e09131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:21.565233 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42f56c-347a-4124-af11-8e1198e09131 admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:21.566127 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42f56c-347a-4124-af11-8e1198e09131 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:21.568030 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42f56c-347a-4124-af11-8e1198e09131 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:21.663177 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1049/2061] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:26:21 2022] POST /identity/v3/users => generated 283 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:21.667102 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a79a342-5968-4a16-8e33-8eb7293f9b99 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:21.677406 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a79a342-5968-4a16-8e33-8eb7293f9b99 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:21.681928 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa3ede9-01bc-4d38-8a1c-cf45ec19b44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:21.682523 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3ede9-01bc-4d38-8a1c-cf45ec19b44c admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:21.682795 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3ede9-01bc-4d38-8a1c-cf45ec19b44c admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:21.683062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3ede9-01bc-4d38-8a1c-cf45ec19b44c admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:21.683826 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa3ede9-01bc-4d38-8a1c-cf45ec19b44c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:21.685478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa3ede9-01bc-4d38-8a1c-cf45ec19b44c admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:21.697120 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1013/2062] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:26:21.698721 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42f56c-347a-4124-af11-8e1198e09131 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:21.700598 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa3ede9-01bc-4d38-8a1c-cf45ec19b44c admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:21.700894 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-dd42f56c-347a-4124-af11-8e1198e09131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:21.714787 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa3ede9-01bc-4d38-8a1c-cf45ec19b44c admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:21.716584 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42f56c-347a-4124-af11-8e1198e09131 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:21.721226 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9b0c2-2985-417d-bc9b-8ffd772a715c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:21.722918 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9b0c2-2985-417d-bc9b-8ffd772a715c admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:21.723369 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9b0c2-2985-417d-bc9b-8ffd772a715c admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:21.723638 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9b0c2-2985-417d-bc9b-8ffd772a715c admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:21.724381 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba9b0c2-2985-417d-bc9b-8ffd772a715c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:21.725990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba9b0c2-2985-417d-bc9b-8ffd772a715c admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:21.728627 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a73e3e-e318-48bd-9470-8aa652c98b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:21.728955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a73e3e-e318-48bd-9470-8aa652c98b73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:21.729253 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a73e3e-e318-48bd-9470-8aa652c98b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:21.729504 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a73e3e-e318-48bd-9470-8aa652c98b73 admin admin] PATH_INFO: `/v3/projects/68e5168b1f42490fb94aecf8e0659d54` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:21.732907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a73e3e-e318-48bd-9470-8aa652c98b73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68e5168b1f42490fb94aecf8e0659d54)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:21.734869 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a73e3e-e318-48bd-9470-8aa652c98b73 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:21.736502 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1014/2063] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:26:21.741360 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9b0c2-2985-417d-bc9b-8ffd772a715c admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:21.751365 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9b0c2-2985-417d-bc9b-8ffd772a715c admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:21.758387 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807fd5a-b1c5-4504-8447-f56f7d2f7a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:21.759118 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807fd5a-b1c5-4504-8447-f56f7d2f7a3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:21.759398 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807fd5a-b1c5-4504-8447-f56f7d2f7a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:21.759671 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807fd5a-b1c5-4504-8447-f56f7d2f7a3a admin admin] PATH_INFO: `/v3/projects/cba3e2e9f37f4924882b775da294e73f/users/b90b57786de94672b199e2854b7eb665/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:21.776069 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6807fd5a-b1c5-4504-8447-f56f7d2f7a3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cba3e2e9f37f4924882b775da294e73f, user_id=b90b57786de94672b199e2854b7eb665, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:21.780825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6807fd5a-b1c5-4504-8447-f56f7d2f7a3a admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:21.848889 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:21.856532 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1015/2064] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:26:21 2022] PUT /identity/v3/projects/cba3e2e9f37f4924882b775da294e73f/users/b90b57786de94672b199e2854b7eb665/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:26:21.869560 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8f280-90c7-42ab-9375-50746fc7a603 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:21.870166 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8f280-90c7-42ab-9375-50746fc7a603 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:21.870445 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8f280-90c7-42ab-9375-50746fc7a603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:21.873228 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-76a73e3e-e318-48bd-9470-8aa652c98b73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68e5168b1f42490fb94aecf8e0659d54', 'request_id': 'req-76a73e3e-e318-48bd-9470-8aa652c98b73'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:21.880812 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-76a73e3e-e318-48bd-9470-8aa652c98b73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68e5168b1f42490fb94aecf8e0659d54', 'request_id': 'req-76a73e3e-e318-48bd-9470-8aa652c98b73'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:21.889002 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-76a73e3e-e318-48bd-9470-8aa652c98b73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68e5168b1f42490fb94aecf8e0659d54', 'request_id': 'req-76a73e3e-e318-48bd-9470-8aa652c98b73'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:21.903319 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-f7c8f280-90c7-42ab-9375-50746fc7a603 None None] MFA Rules not processed for user `b90b57786de94672b199e2854b7eb665`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:21.904728 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f7c8f280-90c7-42ab-9375-50746fc7a603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:21.970055 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1016/2065] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:26:21.972513 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:21.976943 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8f280-90c7-42ab-9375-50746fc7a603 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:21.981011 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-76a73e3e-e318-48bd-9470-8aa652c98b73 admin admin] The token cache is being invalidate because project 68e5168b1f42490fb94aecf8e0659d54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:26:21.981216 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-76a73e3e-e318-48bd-9470-8aa652c98b73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:21.981903 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-76a73e3e-e318-48bd-9470-8aa652c98b73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:21.983613 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1050/2066] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:26:21 2022] DELETE /identity/v3/projects/68e5168b1f42490fb94aecf8e0659d54 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:21.989026 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a135d8e4-2875-40e1-94da-5aef2ab6f206 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:21.989309 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a135d8e4-2875-40e1-94da-5aef2ab6f206 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:21.989575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a135d8e4-2875-40e1-94da-5aef2ab6f206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:22.009631 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-a135d8e4-2875-40e1-94da-5aef2ab6f206 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:22.010850 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-a135d8e4-2875-40e1-94da-5aef2ab6f206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:22.047334 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8f280-90c7-42ab-9375-50746fc7a603 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:22.052748 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e539b8a-3f73-47b9-ad48-302145fd1838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:22.053406 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e539b8a-3f73-47b9-ad48-302145fd1838 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:22.053682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e539b8a-3f73-47b9-ad48-302145fd1838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:22.053949 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e539b8a-3f73-47b9-ad48-302145fd1838 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:22.054608 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e539b8a-3f73-47b9-ad48-302145fd1838 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:22.056542 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e539b8a-3f73-47b9-ad48-302145fd1838 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:22.069553 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1051/2067] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:22.077173 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224bc559-98b7-4f59-ba70-f89babc6fd64 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:22.077462 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224bc559-98b7-4f59-ba70-f89babc6fd64 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:22.077738 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224bc559-98b7-4f59-ba70-f89babc6fd64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:22.097695 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-224bc559-98b7-4f59-ba70-f89babc6fd64 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:22.102827 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-224bc559-98b7-4f59-ba70-f89babc6fd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:22.106735 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1052/2068] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:26:22.110803 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224bc559-98b7-4f59-ba70-f89babc6fd64 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:22.119947 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224bc559-98b7-4f59-ba70-f89babc6fd64 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:22.124527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d9d58-2493-48b3-9660-94d18ca27fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:22.125101 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d9d58-2493-48b3-9660-94d18ca27fd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:22.125378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d9d58-2493-48b3-9660-94d18ca27fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:22.125648 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d9d58-2493-48b3-9660-94d18ca27fd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:22.128845 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d9d58-2493-48b3-9660-94d18ca27fd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:26:22.128845 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d9d58-2493-48b3-9660-94d18ca27fd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:22.128845 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d9d58-2493-48b3-9660-94d18ca27fd1 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:22.139987 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1053/2069] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:26:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:26:22.144885 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d9d58-2493-48b3-9660-94d18ca27fd1 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:22.155924 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d9d58-2493-48b3-9660-94d18ca27fd1 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:22.163435 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1017/2070] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:26:21 2022] POST /identity/v3/projects => generated 367 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:22.166275 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e539b8a-3f73-47b9-ad48-302145fd1838 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:22.167976 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-6e539b8a-3f73-47b9-ad48-302145fd1838 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:22.214378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65666bf-a2f4-4434-9559-ce973a63b223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:22.215003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65666bf-a2f4-4434-9559-ce973a63b223 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:22.215352 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65666bf-a2f4-4434-9559-ce973a63b223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:22.215631 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65666bf-a2f4-4434-9559-ce973a63b223 admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:22.216609 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65666bf-a2f4-4434-9559-ce973a63b223 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:22.218497 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65666bf-a2f4-4434-9559-ce973a63b223 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:22.235693 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e539b8a-3f73-47b9-ad48-302145fd1838 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:22.240988 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b5bf9-9084-48ba-a4e5-ae03c41446da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:22.241615 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b5bf9-9084-48ba-a4e5-ae03c41446da admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:22.241886 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b5bf9-9084-48ba-a4e5-ae03c41446da admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:22.242151 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b5bf9-9084-48ba-a4e5-ae03c41446da admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:22.242803 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686b5bf9-9084-48ba-a4e5-ae03c41446da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:22.249625 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686b5bf9-9084-48ba-a4e5-ae03c41446da admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:22.360554 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1054/2071] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:26:22 2022] POST /identity/v3/projects => generated 369 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:22.367024 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65666bf-a2f4-4434-9559-ce973a63b223 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:22.388852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1018/2072] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:26:22 2022] POST /identity/v3/users => generated 290 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:26:22.400322 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b5bf9-9084-48ba-a4e5-ae03c41446da admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:22.417280 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b5bf9-9084-48ba-a4e5-ae03c41446da admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:22.431448 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65666bf-a2f4-4434-9559-ce973a63b223 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:22.436495 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ddd39-f680-49d6-b0b4-3d72318f5d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:22.437201 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ddd39-f680-49d6-b0b4-3d72318f5d67 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:22.437475 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ddd39-f680-49d6-b0b4-3d72318f5d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:22.437742 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ddd39-f680-49d6-b0b4-3d72318f5d67 admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:22.438403 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42ddd39-f680-49d6-b0b4-3d72318f5d67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:22.441256 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42ddd39-f680-49d6-b0b4-3d72318f5d67 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:22.474826 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b380761-cfdc-4239-9158-ad9cd4c57817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:22.475449 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b380761-cfdc-4239-9158-ad9cd4c57817 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:22.475729 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b380761-cfdc-4239-9158-ad9cd4c57817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:22.476001 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b380761-cfdc-4239-9158-ad9cd4c57817 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:22.476734 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b380761-cfdc-4239-9158-ad9cd4c57817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:22.478314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b380761-cfdc-4239-9158-ad9cd4c57817 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:22.488139 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1019/2073] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:26:22.492297 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b380761-cfdc-4239-9158-ad9cd4c57817 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:22.501839 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b380761-cfdc-4239-9158-ad9cd4c57817 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:22.506927 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb75917-64fb-476f-bb88-b35f098fa24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:22.507593 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb75917-64fb-476f-bb88-b35f098fa24a admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:22.507876 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb75917-64fb-476f-bb88-b35f098fa24a admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:22.508151 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb75917-64fb-476f-bb88-b35f098fa24a admin admin] PATH_INFO: `/v3/projects/c38225867e1a43c78408f8ab1b9983ee/users/9ebf83fb8ec144ada6a8dc80047fa718/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:22.526843 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb75917-64fb-476f-bb88-b35f098fa24a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c38225867e1a43c78408f8ab1b9983ee, user_id=9ebf83fb8ec144ada6a8dc80047fa718, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:22.534950 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb75917-64fb-476f-bb88-b35f098fa24a admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:22.535085 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1055/2074] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:26:22 2022] POST /identity/v3/users => generated 285 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:26:22.536048 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ddd39-f680-49d6-b0b4-3d72318f5d67 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:22.545973 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ddd39-f680-49d6-b0b4-3d72318f5d67 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:22.550604 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197d24b-5ece-4808-b72b-ce621abbd2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:22.551203 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197d24b-5ece-4808-b72b-ce621abbd2fd admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:22.551485 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197d24b-5ece-4808-b72b-ce621abbd2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:22.551858 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197d24b-5ece-4808-b72b-ce621abbd2fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:22.557185 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f197d24b-5ece-4808-b72b-ce621abbd2fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:22.557185 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f197d24b-5ece-4808-b72b-ce621abbd2fd admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:22.564060 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1056/2075] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:26:22.568461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197d24b-5ece-4808-b72b-ce621abbd2fd admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:22.577710 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197d24b-5ece-4808-b72b-ce621abbd2fd admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:22.583242 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcdda36-8669-4238-ba6f-62d428245bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:22.583810 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcdda36-8669-4238-ba6f-62d428245bdc admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:22.584083 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcdda36-8669-4238-ba6f-62d428245bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:22.584352 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcdda36-8669-4238-ba6f-62d428245bdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:22.585026 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcdda36-8669-4238-ba6f-62d428245bdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:22.586589 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcdda36-8669-4238-ba6f-62d428245bdc admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:22.597537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1057/2076] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:26:22.601387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcdda36-8669-4238-ba6f-62d428245bdc admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:22.611607 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcdda36-8669-4238-ba6f-62d428245bdc admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:22.616657 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822e9b2-540b-4484-9373-465b75d18852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:22.617365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822e9b2-540b-4484-9373-465b75d18852 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:22.634868 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822e9b2-540b-4484-9373-465b75d18852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:22.634868 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822e9b2-540b-4484-9373-465b75d18852 admin admin] PATH_INFO: `/v3/projects/580720a3054d477986b3d1feb1f99781/users/d4251382d5d34b5f8fc6037f966f84ee/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:22.635079 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1020/2077] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:26:22 2022] PUT /identity/v3/projects/c38225867e1a43c78408f8ab1b9983ee/users/9ebf83fb8ec144ada6a8dc80047fa718/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:26:22.635185 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b822e9b2-540b-4484-9373-465b75d18852 admin admin] RBAC: Authorizing `identity:create_grant(project_id=580720a3054d477986b3d1feb1f99781, user_id=d4251382d5d34b5f8fc6037f966f84ee, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:22.635985 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b822e9b2-540b-4484-9373-465b75d18852 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:22.637514 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb75917-64fb-476f-bb88-b35f098fa24a admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:22.698320 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb75917-64fb-476f-bb88-b35f098fa24a admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:22.702988 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8700f2-d23e-4157-b7a2-dc36806e2707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:22.703616 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8700f2-d23e-4157-b7a2-dc36806e2707 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:22.703889 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8700f2-d23e-4157-b7a2-dc36806e2707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:22.704154 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8700f2-d23e-4157-b7a2-dc36806e2707 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:22.704858 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8700f2-d23e-4157-b7a2-dc36806e2707 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:22.707096 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8700f2-d23e-4157-b7a2-dc36806e2707 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:22.716910 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1021/2078] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:26:22.721050 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8700f2-d23e-4157-b7a2-dc36806e2707 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:22.725633 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1058/2079] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:26:22 2022] PUT /identity/v3/projects/580720a3054d477986b3d1feb1f99781/users/d4251382d5d34b5f8fc6037f966f84ee/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:26:22.731172 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8700f2-d23e-4157-b7a2-dc36806e2707 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:22.735857 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e4f5b-68b3-43ef-9f6f-e390bde549b3 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:22.736146 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e4f5b-68b3-43ef-9f6f-e390bde549b3 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:22.736439 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e4f5b-68b3-43ef-9f6f-e390bde549b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:22.764664 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-ab9e4f5b-68b3-43ef-9f6f-e390bde549b3 None None] MFA Rules not processed for user `d4251382d5d34b5f8fc6037f966f84ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:22.765875 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-ab9e4f5b-68b3-43ef-9f6f-e390bde549b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:22.788937 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b9e22-421c-4bba-84e3-3f6135f6f215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:22.789673 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b9e22-421c-4bba-84e3-3f6135f6f215 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:22.789887 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b9e22-421c-4bba-84e3-3f6135f6f215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:22.790267 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b9e22-421c-4bba-84e3-3f6135f6f215 admin admin] PATH_INFO: `/v3/projects/c38225867e1a43c78408f8ab1b9983ee/users/9ebf83fb8ec144ada6a8dc80047fa718/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:22.792243 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b9e22-421c-4bba-84e3-3f6135f6f215 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c38225867e1a43c78408f8ab1b9983ee, user_id=9ebf83fb8ec144ada6a8dc80047fa718, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:22.794942 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b9e22-421c-4bba-84e3-3f6135f6f215 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:22.805280 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1022/2080] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:26:22 2022] PUT /identity/v3/projects/c38225867e1a43c78408f8ab1b9983ee/users/9ebf83fb8ec144ada6a8dc80047fa718/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:26:22.813456 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:22.816687 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3dcddc-3473-4469-b12d-d3f32e808be8 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:22.817108 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3dcddc-3473-4469-b12d-d3f32e808be8 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:22.822032 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3dcddc-3473-4469-b12d-d3f32e808be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:22.833261 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1059/2081] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:22.838944 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:22.844916 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e4f5b-68b3-43ef-9f6f-e390bde549b3 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:22.846420 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-ab9e4f5b-68b3-43ef-9f6f-e390bde549b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:22.850042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-aa3dcddc-3473-4469-b12d-d3f32e808be8 None None] MFA Rules not processed for user `9ebf83fb8ec144ada6a8dc80047fa718`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:22.851225 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-aa3dcddc-3473-4469-b12d-d3f32e808be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:22.928731 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e4f5b-68b3-43ef-9f6f-e390bde549b3 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:22.935010 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1023/2082] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:22 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:26:22.998417 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3dcddc-3473-4469-b12d-d3f32e808be8 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:22.998508 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9a7d8f-70ab-49ce-a76f-2b29cb13d2e7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1ce2a2b4eb405b999d3fb756795176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:22.998623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-aa3dcddc-3473-4469-b12d-d3f32e808be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:22.998701 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a7d8f-70ab-49ce-a76f-2b29cb13d2e7 service glance] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:22.998701 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a7d8f-70ab-49ce-a76f-2b29cb13d2e7 service glance] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:22.998701 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a7d8f-70ab-49ce-a76f-2b29cb13d2e7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:23.048822 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9a7d8f-70ab-49ce-a76f-2b29cb13d2e7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:23.048822 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9a7d8f-70ab-49ce-a76f-2b29cb13d2e7 service glance] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:23.050970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3dcddc-3473-4469-b12d-d3f32e808be8 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:23.052380 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1060/2083] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 14:26:23.055062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe09c8-fede-4f6b-915e-a378b0b17fae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:23.055684 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe09c8-fede-4f6b-915e-a378b0b17fae service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:23.055960 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe09c8-fede-4f6b-915e-a378b0b17fae service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:23.056233 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe09c8-fede-4f6b-915e-a378b0b17fae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:23.065491 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe09c8-fede-4f6b-915e-a378b0b17fae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:23.067243 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe09c8-fede-4f6b-915e-a378b0b17fae service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:23.078875 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1024/2084] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:22 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 14:26:23.274429 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:23.669713 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9a7d8f-70ab-49ce-a76f-2b29cb13d2e7 service glance] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:23.679565 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9a7d8f-70ab-49ce-a76f-2b29cb13d2e7 service glance] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:23.683336 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9956741f-c983-4bda-9dea-f751924dc172 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:23.683965 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956741f-c983-4bda-9dea-f751924dc172 service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:23.684244 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956741f-c983-4bda-9dea-f751924dc172 service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:23.684549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956741f-c983-4bda-9dea-f751924dc172 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:23.686299 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-9956741f-c983-4bda-9dea-f751924dc172 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:23.744948 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9956741f-c983-4bda-9dea-f751924dc172 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:23.746377 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9956741f-c983-4bda-9dea-f751924dc172 service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:23.755752 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1061/2085] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:26:23.839769 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:24.136800 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:24.211514 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:24.988208 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:25.079182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:DELETE=16,keystone:SELECT=79,keystone:INSERT=5 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:25.382574 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:25.417081 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:25.462639 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:25.537178 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:25.553944 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:25.696771 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:25.719371 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:25.799156 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:25.931877 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:25.971250 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:26.272152 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:26.300626 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:26.376794 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:26.462598 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:26.490843 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:26.633880 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:32.640508 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe09c8-fede-4f6b-915e-a378b0b17fae service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:32.681919 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9956741f-c983-4bda-9dea-f751924dc172 service cinder] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:32.684128 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-9956741f-c983-4bda-9dea-f751924dc172 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:32.744339 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe09c8-fede-4f6b-915e-a378b0b17fae service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:32.748807 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:32.763561 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a439d764-a28c-426e-b137-30261e06e8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:32.769055 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439d764-a28c-426e-b137-30261e06e8e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:32.769055 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439d764-a28c-426e-b137-30261e06e8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:32.769055 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439d764-a28c-426e-b137-30261e06e8e5 admin admin] PATH_INFO: `/v3/users/b90b57786de94672b199e2854b7eb665` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:32.769055 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a439d764-a28c-426e-b137-30261e06e8e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b90b57786de94672b199e2854b7eb665)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:32.769055 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a439d764-a28c-426e-b137-30261e06e8e5 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:32.773155 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9956741f-c983-4bda-9dea-f751924dc172 service cinder] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:32.779300 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1b614-1062-4c4e-b1d2-23fc608a366c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:32.780368 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1b614-1062-4c4e-b1d2-23fc608a366c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:32.780682 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1b614-1062-4c4e-b1d2-23fc608a366c admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:32.780960 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1b614-1062-4c4e-b1d2-23fc608a366c admin admin] PATH_INFO: `/v3/users/5226fb20ff3c437882f9b8dce2cb83c8` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:32.782140 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b1b614-1062-4c4e-b1d2-23fc608a366c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5226fb20ff3c437882f9b8dce2cb83c8)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:32.784060 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b1b614-1062-4c4e-b1d2-23fc608a366c admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:32.972818 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-67b1b614-1062-4c4e-b1d2-23fc608a366c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5226fb20ff3c437882f9b8dce2cb83c8'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:32.981584 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-67b1b614-1062-4c4e-b1d2-23fc608a366c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5226fb20ff3c437882f9b8dce2cb83c8'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:32.982229 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-67b1b614-1062-4c4e-b1d2-23fc608a366c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5226fb20ff3c437882f9b8dce2cb83c8'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:32.982852 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-67b1b614-1062-4c4e-b1d2-23fc608a366c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5226fb20ff3c437882f9b8dce2cb83c8'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:32.998469 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-67b1b614-1062-4c4e-b1d2-23fc608a366c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5226fb20ff3c437882f9b8dce2cb83c8'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:33.005601 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:33.020891 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a439d764-a28c-426e-b137-30261e06e8e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b90b57786de94672b199e2854b7eb665'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:33.038458 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a439d764-a28c-426e-b137-30261e06e8e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b90b57786de94672b199e2854b7eb665'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:33.084383 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1062/2086] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:26:32 2022] DELETE /identity/v3/users/5226fb20ff3c437882f9b8dce2cb83c8 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:26:33.089584 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6bcaaa-0707-4606-b171-1a285522bede None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:33.089899 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6bcaaa-0707-4606-b171-1a285522bede None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:33.090177 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6bcaaa-0707-4606-b171-1a285522bede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:33.110748 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-6c6bcaaa-0707-4606-b171-1a285522bede None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:33.111950 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-6c6bcaaa-0707-4606-b171-1a285522bede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:33.172777 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1063/2087] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:26:33.176526 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a439d764-a28c-426e-b137-30261e06e8e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b90b57786de94672b199e2854b7eb665'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:33.177266 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a439d764-a28c-426e-b137-30261e06e8e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b90b57786de94672b199e2854b7eb665'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:33.183982 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6bcaaa-0707-4606-b171-1a285522bede None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:33.185413 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-6c6bcaaa-0707-4606-b171-1a285522bede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:33.192095 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a439d764-a28c-426e-b137-30261e06e8e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b90b57786de94672b199e2854b7eb665'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:33.202326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1025/2088] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:26:32 2022] DELETE /identity/v3/users/b90b57786de94672b199e2854b7eb665 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:33.207025 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd11ca9-5986-4cea-87eb-1beaae9c09da None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:33.207182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd11ca9-5986-4cea-87eb-1beaae9c09da None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:33.207438 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd11ca9-5986-4cea-87eb-1beaae9c09da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:33.234422 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-fcd11ca9-5986-4cea-87eb-1beaae9c09da None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:33.237172 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-fcd11ca9-5986-4cea-87eb-1beaae9c09da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:33.253249 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6bcaaa-0707-4606-b171-1a285522bede None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:33.270478 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:33.307348 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1026/2089] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:26:33.311872 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba306a1-2bc0-4884-94e6-80be2984b2e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:33.312544 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba306a1-2bc0-4884-94e6-80be2984b2e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:33.316780 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba306a1-2bc0-4884-94e6-80be2984b2e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:33.316780 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba306a1-2bc0-4884-94e6-80be2984b2e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:33.316780 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-bba306a1-2bc0-4884-94e6-80be2984b2e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:33.317104 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd11ca9-5986-4cea-87eb-1beaae9c09da None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:33.318270 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-fcd11ca9-5986-4cea-87eb-1beaae9c09da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:33.327755 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba306a1-2bc0-4884-94e6-80be2984b2e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:33.329211 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba306a1-2bc0-4884-94e6-80be2984b2e9 service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:33.330016 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd11ca9-5986-4cea-87eb-1beaae9c09da None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:33.333723 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d35a8-8470-40ca-8611-a706fc06aa03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:33.334374 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d35a8-8470-40ca-8611-a706fc06aa03 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:33.335316 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d35a8-8470-40ca-8611-a706fc06aa03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:33.335684 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d35a8-8470-40ca-8611-a706fc06aa03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:33.340380 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1064/2090] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 14:26:33.345950 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d35a8-8470-40ca-8611-a706fc06aa03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:33.347581 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d35a8-8470-40ca-8611-a706fc06aa03 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:33.363874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1027/2091] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 14:26:33.861464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba306a1-2bc0-4884-94e6-80be2984b2e9 service neutron] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:33.863833 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-bba306a1-2bc0-4884-94e6-80be2984b2e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:33.877838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba306a1-2bc0-4884-94e6-80be2984b2e9 service neutron] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:33.884730 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84293e-edd9-4012-b78b-9fdc3ec394f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:33.885761 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84293e-edd9-4012-b78b-9fdc3ec394f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:33.886079 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84293e-edd9-4012-b78b-9fdc3ec394f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:33.886351 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84293e-edd9-4012-b78b-9fdc3ec394f6 admin admin] PATH_INFO: `/v3/projects/f5f6f677923d479b8b9d2b2f7445f374` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:33.887776 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf84293e-edd9-4012-b78b-9fdc3ec394f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5f6f677923d479b8b9d2b2f7445f374)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:33.889834 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf84293e-edd9-4012-b78b-9fdc3ec394f6 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:33.924641 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d35a8-8470-40ca-8611-a706fc06aa03 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:33.926362 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-787d35a8-8470-40ca-8611-a706fc06aa03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:33.942857 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d35a8-8470-40ca-8611-a706fc06aa03 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:33.948264 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0207aa-2163-4ab7-9291-ad8b651ca2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:33.949067 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0207aa-2163-4ab7-9291-ad8b651ca2d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:33.949573 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0207aa-2163-4ab7-9291-ad8b651ca2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:33.949879 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0207aa-2163-4ab7-9291-ad8b651ca2d0 admin admin] PATH_INFO: `/v3/projects/cba3e2e9f37f4924882b775da294e73f` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:33.951015 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0207aa-2163-4ab7-9291-ad8b651ca2d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cba3e2e9f37f4924882b775da294e73f)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:33.953315 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0207aa-2163-4ab7-9291-ad8b651ca2d0 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:34.068263 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-bf84293e-edd9-4012-b78b-9fdc3ec394f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5f6f677923d479b8b9d2b2f7445f374', 'request_id': 'req-bf84293e-edd9-4012-b78b-9fdc3ec394f6'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:34.074776 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-bf84293e-edd9-4012-b78b-9fdc3ec394f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5f6f677923d479b8b9d2b2f7445f374', 'request_id': 'req-bf84293e-edd9-4012-b78b-9fdc3ec394f6'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:34.085218 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-bf84293e-edd9-4012-b78b-9fdc3ec394f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5f6f677923d479b8b9d2b2f7445f374', 'request_id': 'req-bf84293e-edd9-4012-b78b-9fdc3ec394f6'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:34.102794 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-bb0207aa-2163-4ab7-9291-ad8b651ca2d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cba3e2e9f37f4924882b775da294e73f', 'request_id': 'req-bb0207aa-2163-4ab7-9291-ad8b651ca2d0'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:34.109213 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-bb0207aa-2163-4ab7-9291-ad8b651ca2d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cba3e2e9f37f4924882b775da294e73f', 'request_id': 'req-bb0207aa-2163-4ab7-9291-ad8b651ca2d0'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:34.179556 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-bf84293e-edd9-4012-b78b-9fdc3ec394f6 admin admin] The token cache is being invalidate because project f5f6f677923d479b8b9d2b2f7445f374 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:26:34.180173 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-bf84293e-edd9-4012-b78b-9fdc3ec394f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:34.180868 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-bf84293e-edd9-4012-b78b-9fdc3ec394f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:34.182482 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1065/2092] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:26:33 2022] DELETE /identity/v3/projects/f5f6f677923d479b8b9d2b2f7445f374 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:34.186479 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84293e-edd9-4012-b78b-9fdc3ec394f6 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:34.187811 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-bf84293e-edd9-4012-b78b-9fdc3ec394f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:34.232695 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-bb0207aa-2163-4ab7-9291-ad8b651ca2d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cba3e2e9f37f4924882b775da294e73f', 'request_id': 'req-bb0207aa-2163-4ab7-9291-ad8b651ca2d0'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:34.245331 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-bb0207aa-2163-4ab7-9291-ad8b651ca2d0 admin admin] The token cache is being invalidate because project cba3e2e9f37f4924882b775da294e73f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:26:34.245617 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-bb0207aa-2163-4ab7-9291-ad8b651ca2d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:34.246249 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-bb0207aa-2163-4ab7-9291-ad8b651ca2d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:34.247786 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1028/2093] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:26:33 2022] DELETE /identity/v3/projects/cba3e2e9f37f4924882b775da294e73f => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:26:34.251659 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0207aa-2163-4ab7-9291-ad8b651ca2d0 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:34.253193 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-bb0207aa-2163-4ab7-9291-ad8b651ca2d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:34.260326 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84293e-edd9-4012-b78b-9fdc3ec394f6 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:34.289741 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:34.311895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:34.322190 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5929d707-8f21-4f0a-8639-0f4a3702016e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:34.322956 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929d707-8f21-4f0a-8639-0f4a3702016e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:34.325656 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929d707-8f21-4f0a-8639-0f4a3702016e admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:34.325656 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929d707-8f21-4f0a-8639-0f4a3702016e admin admin] PATH_INFO: `/v3/users/e3cdc40e2a1842519d14046be74b3f78` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:34.325656 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5929d707-8f21-4f0a-8639-0f4a3702016e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3cdc40e2a1842519d14046be74b3f78)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:34.326211 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0207aa-2163-4ab7-9291-ad8b651ca2d0 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:34.327421 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5929d707-8f21-4f0a-8639-0f4a3702016e admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:34.331085 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c022c20c-3d3b-466d-bb5d-8b048b60abd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:34.331718 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c022c20c-3d3b-466d-bb5d-8b048b60abd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:34.331988 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c022c20c-3d3b-466d-bb5d-8b048b60abd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:34.332271 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c022c20c-3d3b-466d-bb5d-8b048b60abd9 admin admin] PATH_INFO: `/v3/users/9ebf83fb8ec144ada6a8dc80047fa718` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:34.333535 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c022c20c-3d3b-466d-bb5d-8b048b60abd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ebf83fb8ec144ada6a8dc80047fa718)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:34.336323 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c022c20c-3d3b-466d-bb5d-8b048b60abd9 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:34.510669 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c022c20c-3d3b-466d-bb5d-8b048b60abd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ebf83fb8ec144ada6a8dc80047fa718'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:34.512565 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-5929d707-8f21-4f0a-8639-0f4a3702016e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3cdc40e2a1842519d14046be74b3f78'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:34.526180 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-5929d707-8f21-4f0a-8639-0f4a3702016e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3cdc40e2a1842519d14046be74b3f78'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:34.527165 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-5929d707-8f21-4f0a-8639-0f4a3702016e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3cdc40e2a1842519d14046be74b3f78'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:34.527806 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-5929d707-8f21-4f0a-8639-0f4a3702016e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3cdc40e2a1842519d14046be74b3f78'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:34.535517 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c022c20c-3d3b-466d-bb5d-8b048b60abd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ebf83fb8ec144ada6a8dc80047fa718'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:34.541172 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-5929d707-8f21-4f0a-8639-0f4a3702016e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3cdc40e2a1842519d14046be74b3f78'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:34.608662 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c022c20c-3d3b-466d-bb5d-8b048b60abd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ebf83fb8ec144ada6a8dc80047fa718'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:34.609310 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c022c20c-3d3b-466d-bb5d-8b048b60abd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ebf83fb8ec144ada6a8dc80047fa718'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:34.616797 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c022c20c-3d3b-466d-bb5d-8b048b60abd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ebf83fb8ec144ada6a8dc80047fa718'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:34.622740 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1029/2094] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:26:34 2022] DELETE /identity/v3/users/9ebf83fb8ec144ada6a8dc80047fa718 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:26:34.683390 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1066/2095] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:26:34 2022] DELETE /identity/v3/users/e3cdc40e2a1842519d14046be74b3f78 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:26:35.291554 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c022c20c-3d3b-466d-bb5d-8b048b60abd9 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:35.295799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c022c20c-3d3b-466d-bb5d-8b048b60abd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:35.297629 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5929d707-8f21-4f0a-8639-0f4a3702016e admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:35.298819 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-5929d707-8f21-4f0a-8639-0f4a3702016e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:35.380127 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c022c20c-3d3b-466d-bb5d-8b048b60abd9 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:35.380625 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5929d707-8f21-4f0a-8639-0f4a3702016e admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:35.385618 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e7b11-6df2-418d-873d-19b6bc88cc2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:35.386278 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e7b11-6df2-418d-873d-19b6bc88cc2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:35.386557 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e7b11-6df2-418d-873d-19b6bc88cc2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:35.386835 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e7b11-6df2-418d-873d-19b6bc88cc2d admin admin] PATH_INFO: `/v3/projects/4d675dd935874f619876e2039ad5634b` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:35.388063 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910e7b11-6df2-418d-873d-19b6bc88cc2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d675dd935874f619876e2039ad5634b)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:35.389411 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b79c36-e59b-44ec-b6a9-628682742367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:35.389971 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910e7b11-6df2-418d-873d-19b6bc88cc2d admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:35.391826 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b79c36-e59b-44ec-b6a9-628682742367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:35.392150 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b79c36-e59b-44ec-b6a9-628682742367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:35.392588 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b79c36-e59b-44ec-b6a9-628682742367 admin admin] PATH_INFO: `/v3/projects/c38225867e1a43c78408f8ab1b9983ee` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:35.399347 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b79c36-e59b-44ec-b6a9-628682742367 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c38225867e1a43c78408f8ab1b9983ee)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:35.399347 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b79c36-e59b-44ec-b6a9-628682742367 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:35.487183 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-910e7b11-6df2-418d-873d-19b6bc88cc2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d675dd935874f619876e2039ad5634b', 'request_id': 'req-910e7b11-6df2-418d-873d-19b6bc88cc2d'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:35.494562 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-910e7b11-6df2-418d-873d-19b6bc88cc2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d675dd935874f619876e2039ad5634b', 'request_id': 'req-910e7b11-6df2-418d-873d-19b6bc88cc2d'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:35.500243 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-910e7b11-6df2-418d-873d-19b6bc88cc2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d675dd935874f619876e2039ad5634b', 'request_id': 'req-910e7b11-6df2-418d-873d-19b6bc88cc2d'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:35.502591 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-04b79c36-e59b-44ec-b6a9-628682742367 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c38225867e1a43c78408f8ab1b9983ee', 'request_id': 'req-04b79c36-e59b-44ec-b6a9-628682742367'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:35.503989 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:35.510018 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-04b79c36-e59b-44ec-b6a9-628682742367 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c38225867e1a43c78408f8ab1b9983ee', 'request_id': 'req-04b79c36-e59b-44ec-b6a9-628682742367'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:35.549088 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:35.553059 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-910e7b11-6df2-418d-873d-19b6bc88cc2d admin admin] The token cache is being invalidate because project 4d675dd935874f619876e2039ad5634b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:26:35.554216 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-910e7b11-6df2-418d-873d-19b6bc88cc2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:35.554663 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-910e7b11-6df2-418d-873d-19b6bc88cc2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:35.556534 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1067/2096] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:26:35 2022] DELETE /identity/v3/projects/4d675dd935874f619876e2039ad5634b => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:35.560941 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e7b11-6df2-418d-873d-19b6bc88cc2d admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:35.562309 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-910e7b11-6df2-418d-873d-19b6bc88cc2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:35.615986 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-04b79c36-e59b-44ec-b6a9-628682742367 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c38225867e1a43c78408f8ab1b9983ee', 'request_id': 'req-04b79c36-e59b-44ec-b6a9-628682742367'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:35.627203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-04b79c36-e59b-44ec-b6a9-628682742367 admin admin] The token cache is being invalidate because project c38225867e1a43c78408f8ab1b9983ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:26:35.627475 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-04b79c36-e59b-44ec-b6a9-628682742367 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:35.628095 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-04b79c36-e59b-44ec-b6a9-628682742367 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:35.629306 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e7b11-6df2-418d-873d-19b6bc88cc2d admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:35.629655 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1030/2097] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:26:35 2022] DELETE /identity/v3/projects/c38225867e1a43c78408f8ab1b9983ee => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:35.635195 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069dda0-fd5c-49e6-8fea-13dd6b436966 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:35.635475 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069dda0-fd5c-49e6-8fea-13dd6b436966 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:35.635751 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069dda0-fd5c-49e6-8fea-13dd6b436966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:35.655493 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-5069dda0-fd5c-49e6-8fea-13dd6b436966 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:35.656725 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5069dda0-fd5c-49e6-8fea-13dd6b436966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:35.692090 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189d9e2-b2f8-4c11-ae87-fd55d1ebf065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:35.692775 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189d9e2-b2f8-4c11-ae87-fd55d1ebf065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:35.693056 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189d9e2-b2f8-4c11-ae87-fd55d1ebf065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:35.693332 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189d9e2-b2f8-4c11-ae87-fd55d1ebf065 admin admin] PATH_INFO: `/v3/users/3d65855b2cac425e845f57430efb204d` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:35.695421 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a189d9e2-b2f8-4c11-ae87-fd55d1ebf065 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d65855b2cac425e845f57430efb204d)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:35.698291 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a189d9e2-b2f8-4c11-ae87-fd55d1ebf065 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:35.717572 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1031/2098] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:26:35.727135 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1cbdd7-5092-4821-afdc-56634d4d84e5 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:35.727430 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1cbdd7-5092-4821-afdc-56634d4d84e5 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:35.727706 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1cbdd7-5092-4821-afdc-56634d4d84e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:35.747530 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-ff1cbdd7-5092-4821-afdc-56634d4d84e5 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:35.748767 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ff1cbdd7-5092-4821-afdc-56634d4d84e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:35.756757 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1032/2099] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:26:35.760787 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1cbdd7-5092-4821-afdc-56634d4d84e5 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:35.770400 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1cbdd7-5092-4821-afdc-56634d4d84e5 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:35.776194 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5875810-98e5-49aa-bcbc-9f866502c56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:35.777434 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5875810-98e5-49aa-bcbc-9f866502c56d admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:35.777711 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5875810-98e5-49aa-bcbc-9f866502c56d admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:35.777978 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5875810-98e5-49aa-bcbc-9f866502c56d admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:35.778958 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5875810-98e5-49aa-bcbc-9f866502c56d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:26:35.779264 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5875810-98e5-49aa-bcbc-9f866502c56d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:35.780986 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5875810-98e5-49aa-bcbc-9f866502c56d admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:35.793975 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1033/2100] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:26:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:26:35.799565 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5875810-98e5-49aa-bcbc-9f866502c56d admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:35.809824 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5875810-98e5-49aa-bcbc-9f866502c56d admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:35.814795 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62737a0e-f076-4af7-bb0b-e3080a682665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:35.815403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62737a0e-f076-4af7-bb0b-e3080a682665 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:35.815681 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62737a0e-f076-4af7-bb0b-e3080a682665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:35.815983 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62737a0e-f076-4af7-bb0b-e3080a682665 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:35.816643 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62737a0e-f076-4af7-bb0b-e3080a682665 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:35.818679 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62737a0e-f076-4af7-bb0b-e3080a682665 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:35.846688 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-a189d9e2-b2f8-4c11-ae87-fd55d1ebf065 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d65855b2cac425e845f57430efb204d'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:35.856847 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-a189d9e2-b2f8-4c11-ae87-fd55d1ebf065 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d65855b2cac425e845f57430efb204d'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:35.857747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-a189d9e2-b2f8-4c11-ae87-fd55d1ebf065 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d65855b2cac425e845f57430efb204d'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:35.858519 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-a189d9e2-b2f8-4c11-ae87-fd55d1ebf065 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d65855b2cac425e845f57430efb204d'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:35.874026 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-a189d9e2-b2f8-4c11-ae87-fd55d1ebf065 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d65855b2cac425e845f57430efb204d'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:35.892247 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1034/2101] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:26:35 2022] POST /identity/v3/projects => generated 383 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:35.897586 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62737a0e-f076-4af7-bb0b-e3080a682665 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:35.899468 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-62737a0e-f076-4af7-bb0b-e3080a682665 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:35.936680 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1068/2102] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:26:35 2022] DELETE /identity/v3/users/3d65855b2cac425e845f57430efb204d => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:26:35.937905 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189d9e2-b2f8-4c11-ae87-fd55d1ebf065 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:35.939485 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-a189d9e2-b2f8-4c11-ae87-fd55d1ebf065 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:35.984869 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:35.989140 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62737a0e-f076-4af7-bb0b-e3080a682665 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:36.034550 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189d9e2-b2f8-4c11-ae87-fd55d1ebf065 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:36.041828 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32fa77-464d-42a9-b069-439ceb8a3af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:36.042939 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32fa77-464d-42a9-b069-439ceb8a3af8 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:36.043429 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32fa77-464d-42a9-b069-439ceb8a3af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:36.043918 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32fa77-464d-42a9-b069-439ceb8a3af8 admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:36.044815 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b32fa77-464d-42a9-b069-439ceb8a3af8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:36.047120 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b32fa77-464d-42a9-b069-439ceb8a3af8 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:36.052839 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:36.075798 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1034bace-7e0a-4061-afb3-7ea10c008e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:36.076570 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1034bace-7e0a-4061-afb3-7ea10c008e84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:36.076959 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1034bace-7e0a-4061-afb3-7ea10c008e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:36.077242 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1034bace-7e0a-4061-afb3-7ea10c008e84 admin admin] PATH_INFO: `/v3/users/9c11b7ea92f44d5f98dacd3de11ebacf` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:36.078443 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1034bace-7e0a-4061-afb3-7ea10c008e84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c11b7ea92f44d5f98dacd3de11ebacf)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:36.080392 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1034bace-7e0a-4061-afb3-7ea10c008e84 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:36.151642 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1069/2103] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:26:35 2022] POST /identity/v3/users => generated 292 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:36.152669 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32fa77-464d-42a9-b069-439ceb8a3af8 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:36.168052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32fa77-464d-42a9-b069-439ceb8a3af8 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:36.173398 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1463a5e4-d301-4bcd-b027-07d862a0c03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:36.174230 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463a5e4-d301-4bcd-b027-07d862a0c03c admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:36.174561 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463a5e4-d301-4bcd-b027-07d862a0c03c admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:36.174821 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463a5e4-d301-4bcd-b027-07d862a0c03c admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:36.175526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1463a5e4-d301-4bcd-b027-07d862a0c03c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:36.177165 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1463a5e4-d301-4bcd-b027-07d862a0c03c admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:36.191594 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1070/2104] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:26:36.192721 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1463a5e4-d301-4bcd-b027-07d862a0c03c admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:36.212283 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1463a5e4-d301-4bcd-b027-07d862a0c03c admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:36.219248 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f183900-d648-4b73-99c5-93183b6d0df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:36.219974 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f183900-d648-4b73-99c5-93183b6d0df3 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:36.220448 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f183900-d648-4b73-99c5-93183b6d0df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:36.220731 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f183900-d648-4b73-99c5-93183b6d0df3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:36.221674 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f183900-d648-4b73-99c5-93183b6d0df3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:36.223759 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f183900-d648-4b73-99c5-93183b6d0df3 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:36.247639 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1071/2105] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:36 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:26:36.249768 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f183900-d648-4b73-99c5-93183b6d0df3 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:36.261322 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1034bace-7e0a-4061-afb3-7ea10c008e84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c11b7ea92f44d5f98dacd3de11ebacf'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:36.277724 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f183900-d648-4b73-99c5-93183b6d0df3 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:36.283860 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346d453-e763-453d-8f4c-27187175e113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:36.285061 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346d453-e763-453d-8f4c-27187175e113 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:36.286751 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346d453-e763-453d-8f4c-27187175e113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:36.291519 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346d453-e763-453d-8f4c-27187175e113 admin admin] PATH_INFO: `/v3/projects/86da919b98164f8791254e988d13bd80/users/9322ed807f0b4b6c8f5bd13ad04620a4/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:36.291664 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1034bace-7e0a-4061-afb3-7ea10c008e84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c11b7ea92f44d5f98dacd3de11ebacf'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:36.307848 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6346d453-e763-453d-8f4c-27187175e113 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86da919b98164f8791254e988d13bd80, user_id=9322ed807f0b4b6c8f5bd13ad04620a4, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:36.313303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6346d453-e763-453d-8f4c-27187175e113 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:36.369934 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1034bace-7e0a-4061-afb3-7ea10c008e84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c11b7ea92f44d5f98dacd3de11ebacf'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:36.370596 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1034bace-7e0a-4061-afb3-7ea10c008e84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c11b7ea92f44d5f98dacd3de11ebacf'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:36.378993 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1034bace-7e0a-4061-afb3-7ea10c008e84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c11b7ea92f44d5f98dacd3de11ebacf'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:36.386871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1035/2106] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:26:35 2022] DELETE /identity/v3/users/9c11b7ea92f44d5f98dacd3de11ebacf => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:36.391340 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870eb93b-a74d-40ff-a30b-a57e666d26c1 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:36.391791 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870eb93b-a74d-40ff-a30b-a57e666d26c1 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:36.392170 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870eb93b-a74d-40ff-a30b-a57e666d26c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:36.418829 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1072/2107] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:26:36 2022] PUT /identity/v3/projects/86da919b98164f8791254e988d13bd80/users/9322ed807f0b4b6c8f5bd13ad04620a4/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:36.421779 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05269a0-117b-4e2b-9651-a2b12a8db9be None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:36.422069 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05269a0-117b-4e2b-9651-a2b12a8db9be None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:36.423470 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05269a0-117b-4e2b-9651-a2b12a8db9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:36.458527 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-870eb93b-a74d-40ff-a30b-a57e666d26c1 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:36.460044 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-870eb93b-a74d-40ff-a30b-a57e666d26c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:36.524018 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-f05269a0-117b-4e2b-9651-a2b12a8db9be None None] MFA Rules not processed for user `9322ed807f0b4b6c8f5bd13ad04620a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:36.524018 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-f05269a0-117b-4e2b-9651-a2b12a8db9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:36.627927 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1036/2108] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:26:36.629343 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870eb93b-a74d-40ff-a30b-a57e666d26c1 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:36.632530 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-870eb93b-a74d-40ff-a30b-a57e666d26c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:36.637967 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1073/2109] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:36.642504 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05269a0-117b-4e2b-9651-a2b12a8db9be None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:36.644146 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-f05269a0-117b-4e2b-9651-a2b12a8db9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:36.651650 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870eb93b-a74d-40ff-a30b-a57e666d26c1 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:36.656682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d558f-aa6f-4e72-a251-7aabc3c9cc2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:36.657343 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d558f-aa6f-4e72-a251-7aabc3c9cc2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:36.657621 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d558f-aa6f-4e72-a251-7aabc3c9cc2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:36.657891 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d558f-aa6f-4e72-a251-7aabc3c9cc2f admin admin] PATH_INFO: `/v3/projects/697f0b95d9af4ddf952c83b1cd8caa44` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:36.659305 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4d558f-aa6f-4e72-a251-7aabc3c9cc2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=697f0b95d9af4ddf952c83b1cd8caa44)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:36.661971 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4d558f-aa6f-4e72-a251-7aabc3c9cc2f admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:36.728516 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05269a0-117b-4e2b-9651-a2b12a8db9be None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:36.732264 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb982d6-af73-429e-bc09-7c92345d4c73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:36.733063 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb982d6-af73-429e-bc09-7c92345d4c73 service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:36.733351 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb982d6-af73-429e-bc09-7c92345d4c73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:36.733626 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb982d6-af73-429e-bc09-7c92345d4c73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:36.745949 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb982d6-af73-429e-bc09-7c92345d4c73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:36.747507 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb982d6-af73-429e-bc09-7c92345d4c73 service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:36.757508 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1074/2110] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:26:36.758297 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb982d6-af73-429e-bc09-7c92345d4c73 service neutron] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:36.759632 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-eeb982d6-af73-429e-bc09-7c92345d4c73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:36.767255 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:36.772538 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb982d6-af73-429e-bc09-7c92345d4c73 service neutron] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:36.777293 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a89b25-af72-426f-b0c0-a887a49c3c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:36.777959 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a89b25-af72-426f-b0c0-a887a49c3c6c admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:36.778179 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a89b25-af72-426f-b0c0-a887a49c3c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:36.778467 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a89b25-af72-426f-b0c0-a887a49c3c6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:36.779111 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a89b25-af72-426f-b0c0-a887a49c3c6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:36.781051 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a89b25-af72-426f-b0c0-a887a49c3c6c admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:36.810946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9e4d558f-aa6f-4e72-a251-7aabc3c9cc2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '697f0b95d9af4ddf952c83b1cd8caa44', 'request_id': 'req-9e4d558f-aa6f-4e72-a251-7aabc3c9cc2f'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:36.817817 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9e4d558f-aa6f-4e72-a251-7aabc3c9cc2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '697f0b95d9af4ddf952c83b1cd8caa44', 'request_id': 'req-9e4d558f-aa6f-4e72-a251-7aabc3c9cc2f'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:36.822529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:36.884523 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9e4d558f-aa6f-4e72-a251-7aabc3c9cc2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '697f0b95d9af4ddf952c83b1cd8caa44', 'request_id': 'req-9e4d558f-aa6f-4e72-a251-7aabc3c9cc2f'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:36.898597 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1075/2111] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:26:36 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:36.898713 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9e4d558f-aa6f-4e72-a251-7aabc3c9cc2f admin admin] The token cache is being invalidate because project 697f0b95d9af4ddf952c83b1cd8caa44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:26:36.898713 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9e4d558f-aa6f-4e72-a251-7aabc3c9cc2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:36.898713 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9e4d558f-aa6f-4e72-a251-7aabc3c9cc2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:36.900911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a89b25-af72-426f-b0c0-a887a49c3c6c admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:36.902704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-58a89b25-af72-426f-b0c0-a887a49c3c6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:36.903349 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1037/2112] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:26:36 2022] DELETE /identity/v3/projects/697f0b95d9af4ddf952c83b1cd8caa44 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:36.904813 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff999a58-7364-42a5-91fa-fbc05e5859de None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:36.905101 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff999a58-7364-42a5-91fa-fbc05e5859de None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:36.905544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff999a58-7364-42a5-91fa-fbc05e5859de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:36.930958 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-ff999a58-7364-42a5-91fa-fbc05e5859de None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:36.932365 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ff999a58-7364-42a5-91fa-fbc05e5859de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:37.005572 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a89b25-af72-426f-b0c0-a887a49c3c6c admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:37.011531 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c21b0-d629-488d-ad9a-771f0be7b671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:37.012373 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c21b0-d629-488d-ad9a-771f0be7b671 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:37.012373 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c21b0-d629-488d-ad9a-771f0be7b671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:37.012797 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c21b0-d629-488d-ad9a-771f0be7b671 admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:37.013564 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c21b0-d629-488d-ad9a-771f0be7b671 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:37.015562 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c21b0-d629-488d-ad9a-771f0be7b671 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:37.034430 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1038/2113] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:37.044890 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa40614-9da8-4240-a122-f2dbd2140f6e None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:37.045169 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa40614-9da8-4240-a122-f2dbd2140f6e None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:37.045440 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa40614-9da8-4240-a122-f2dbd2140f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:37.066904 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-7fa40614-9da8-4240-a122-f2dbd2140f6e None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:37.068217 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-7fa40614-9da8-4240-a122-f2dbd2140f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:37.077527 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1039/2114] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:26:37.081664 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa40614-9da8-4240-a122-f2dbd2140f6e None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:37.092442 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa40614-9da8-4240-a122-f2dbd2140f6e None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:37.100989 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f949156d-a1f5-4333-8f2d-538267c57733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:37.101620 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f949156d-a1f5-4333-8f2d-538267c57733 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:37.101894 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f949156d-a1f5-4333-8f2d-538267c57733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:37.102165 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f949156d-a1f5-4333-8f2d-538267c57733 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:37.102857 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f949156d-a1f5-4333-8f2d-538267c57733 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:26:37.103154 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f949156d-a1f5-4333-8f2d-538267c57733 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:37.104764 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f949156d-a1f5-4333-8f2d-538267c57733 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:37.116401 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1040/2115] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:26:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:26:37.120479 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1076/2116] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:26:36 2022] POST /identity/v3/users => generated 293 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:26:37.121361 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f949156d-a1f5-4333-8f2d-538267c57733 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:37.122694 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c21b0-d629-488d-ad9a-771f0be7b671 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:37.131472 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f949156d-a1f5-4333-8f2d-538267c57733 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:37.132936 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c21b0-d629-488d-ad9a-771f0be7b671 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:37.136571 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a29e00-52b2-44c8-929b-a3ca3fb9cc65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:37.137184 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a29e00-52b2-44c8-929b-a3ca3fb9cc65 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:37.137463 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a29e00-52b2-44c8-929b-a3ca3fb9cc65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:37.137736 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a29e00-52b2-44c8-929b-a3ca3fb9cc65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:37.138527 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a29e00-52b2-44c8-929b-a3ca3fb9cc65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:37.138874 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28453e4-e513-46d9-90ac-5d81036c3712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:37.139480 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28453e4-e513-46d9-90ac-5d81036c3712 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:37.139870 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28453e4-e513-46d9-90ac-5d81036c3712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:37.140144 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28453e4-e513-46d9-90ac-5d81036c3712 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:37.140697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a29e00-52b2-44c8-929b-a3ca3fb9cc65 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:37.140940 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28453e4-e513-46d9-90ac-5d81036c3712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:37.147460 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28453e4-e513-46d9-90ac-5d81036c3712 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:37.154042 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1077/2117] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:26:37.161323 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28453e4-e513-46d9-90ac-5d81036c3712 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:37.176006 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28453e4-e513-46d9-90ac-5d81036c3712 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:37.178971 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e37aa-9fcd-46e3-925c-178cd11c1f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:37.179755 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e37aa-9fcd-46e3-925c-178cd11c1f99 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:37.180081 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e37aa-9fcd-46e3-925c-178cd11c1f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:37.180574 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e37aa-9fcd-46e3-925c-178cd11c1f99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:37.181448 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7e37aa-9fcd-46e3-925c-178cd11c1f99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:37.182985 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7e37aa-9fcd-46e3-925c-178cd11c1f99 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:37.197580 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1078/2118] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:26:37.197580 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e37aa-9fcd-46e3-925c-178cd11c1f99 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:37.199285 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-ac7e37aa-9fcd-46e3-925c-178cd11c1f99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:37.213649 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e37aa-9fcd-46e3-925c-178cd11c1f99 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:37.221122 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6273f1-1f4c-4734-a6b2-5f40dcc37d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:37.221901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6273f1-1f4c-4734-a6b2-5f40dcc37d00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:37.222760 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6273f1-1f4c-4734-a6b2-5f40dcc37d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:37.224038 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6273f1-1f4c-4734-a6b2-5f40dcc37d00 admin admin] PATH_INFO: `/v3/projects/ade6d833426847fcb9bf2ee7227f4c63` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:37.227135 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6273f1-1f4c-4734-a6b2-5f40dcc37d00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ade6d833426847fcb9bf2ee7227f4c63)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:37.229136 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6273f1-1f4c-4734-a6b2-5f40dcc37d00 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:37.241683 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1041/2119] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:26:37 2022] POST /identity/v3/projects => generated 385 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:37.242694 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a29e00-52b2-44c8-929b-a3ca3fb9cc65 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:37.306320 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a29e00-52b2-44c8-929b-a3ca3fb9cc65 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:37.311218 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537dbd77-2af5-4541-ba6f-84f16772d4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:37.311945 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537dbd77-2af5-4541-ba6f-84f16772d4e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:37.312548 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537dbd77-2af5-4541-ba6f-84f16772d4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:37.313006 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537dbd77-2af5-4541-ba6f-84f16772d4e4 admin admin] PATH_INFO: `/v3/projects/b496b1efd25349f3908458c9d212dbcf/users/229dbb2594e643098e9123a8a13f3019/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:37.327935 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537dbd77-2af5-4541-ba6f-84f16772d4e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b496b1efd25349f3908458c9d212dbcf, user_id=229dbb2594e643098e9123a8a13f3019, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:37.330723 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537dbd77-2af5-4541-ba6f-84f16772d4e4 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:37.422135 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1042/2120] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:26:37 2022] PUT /identity/v3/projects/b496b1efd25349f3908458c9d212dbcf/users/229dbb2594e643098e9123a8a13f3019/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:37.422939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537dbd77-2af5-4541-ba6f-84f16772d4e4 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:37.433612 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9b6273f1-1f4c-4734-a6b2-5f40dcc37d00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ade6d833426847fcb9bf2ee7227f4c63', 'request_id': 'req-9b6273f1-1f4c-4734-a6b2-5f40dcc37d00'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:37.439722 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9b6273f1-1f4c-4734-a6b2-5f40dcc37d00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ade6d833426847fcb9bf2ee7227f4c63', 'request_id': 'req-9b6273f1-1f4c-4734-a6b2-5f40dcc37d00'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:37.446000 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9b6273f1-1f4c-4734-a6b2-5f40dcc37d00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ade6d833426847fcb9bf2ee7227f4c63', 'request_id': 'req-9b6273f1-1f4c-4734-a6b2-5f40dcc37d00'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:37.481046 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:37.492301 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537dbd77-2af5-4541-ba6f-84f16772d4e4 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:37.501481 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9b6273f1-1f4c-4734-a6b2-5f40dcc37d00 admin admin] The token cache is being invalidate because project ade6d833426847fcb9bf2ee7227f4c63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:26:37.501983 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9b6273f1-1f4c-4734-a6b2-5f40dcc37d00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:37.502934 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9b6273f1-1f4c-4734-a6b2-5f40dcc37d00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:37.516440 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1079/2121] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:26:37 2022] DELETE /identity/v3/projects/ade6d833426847fcb9bf2ee7227f4c63 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:37.516440 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b574568-d109-455e-8f73-90d50392c566 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:37.516440 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b574568-d109-455e-8f73-90d50392c566 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:37.516440 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b574568-d109-455e-8f73-90d50392c566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:37.565807 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-3b574568-d109-455e-8f73-90d50392c566 None None] MFA Rules not processed for user `229dbb2594e643098e9123a8a13f3019`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:37.572864 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-3b574568-d109-455e-8f73-90d50392c566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:37.616865 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:37.654918 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa0be3-7bbe-4f4a-93a4-22e5ae813bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:37.655852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa0be3-7bbe-4f4a-93a4-22e5ae813bf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:37.656133 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa0be3-7bbe-4f4a-93a4-22e5ae813bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:37.656406 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa0be3-7bbe-4f4a-93a4-22e5ae813bf4 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:37.657135 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aa0be3-7bbe-4f4a-93a4-22e5ae813bf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:37.660255 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aa0be3-7bbe-4f4a-93a4-22e5ae813bf4 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:37.712051 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1080/2122] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:37.716371 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b574568-d109-455e-8f73-90d50392c566 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:37.718191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-3b574568-d109-455e-8f73-90d50392c566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:37.732833 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b574568-d109-455e-8f73-90d50392c566 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:37.737697 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb80618e-adad-4f38-99de-a3b126fec498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:37.738068 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80618e-adad-4f38-99de-a3b126fec498 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:37.738364 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80618e-adad-4f38-99de-a3b126fec498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:37.738635 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80618e-adad-4f38-99de-a3b126fec498 admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:37.739336 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb80618e-adad-4f38-99de-a3b126fec498 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:37.741307 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb80618e-adad-4f38-99de-a3b126fec498 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:37.792446 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1043/2123] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:26:37 2022] POST /identity/v3/users => generated 293 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:37.795341 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa0be3-7bbe-4f4a-93a4-22e5ae813bf4 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:37.797133 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-30aa0be3-7bbe-4f4a-93a4-22e5ae813bf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:37.812946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa0be3-7bbe-4f4a-93a4-22e5ae813bf4 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:37.818032 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8d448-3c02-41ef-a73e-ffe30c170ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:37.819270 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8d448-3c02-41ef-a73e-ffe30c170ac2 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:37.819671 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8d448-3c02-41ef-a73e-ffe30c170ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:37.819944 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8d448-3c02-41ef-a73e-ffe30c170ac2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:37.820655 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8d448-3c02-41ef-a73e-ffe30c170ac2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:37.822238 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8d448-3c02-41ef-a73e-ffe30c170ac2 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:37.835294 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1044/2124] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:26:37.836360 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8d448-3c02-41ef-a73e-ffe30c170ac2 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:37.846446 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8d448-3c02-41ef-a73e-ffe30c170ac2 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:37.852130 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48452d83-8529-4b2f-94c7-078bdb6e3f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:37.852772 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48452d83-8529-4b2f-94c7-078bdb6e3f7e admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:37.853052 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48452d83-8529-4b2f-94c7-078bdb6e3f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:37.853345 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48452d83-8529-4b2f-94c7-078bdb6e3f7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:37.854042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48452d83-8529-4b2f-94c7-078bdb6e3f7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:37.855659 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48452d83-8529-4b2f-94c7-078bdb6e3f7e admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:37.872300 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1081/2125] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:26:37 2022] POST /identity/v3/projects => generated 381 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:26:37.876013 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb80618e-adad-4f38-99de-a3b126fec498 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:37.877041 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1045/2126] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:37 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:26:37.878782 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48452d83-8529-4b2f-94c7-078bdb6e3f7e admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:37.899237 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48452d83-8529-4b2f-94c7-078bdb6e3f7e admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:37.960314 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb80618e-adad-4f38-99de-a3b126fec498 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:37.967042 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dab1b0-8070-453a-a15e-b2e553fdc52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:37.967671 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dab1b0-8070-453a-a15e-b2e553fdc52f admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:37.967948 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dab1b0-8070-453a-a15e-b2e553fdc52f admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:37.968346 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dab1b0-8070-453a-a15e-b2e553fdc52f admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:37.969392 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dab1b0-8070-453a-a15e-b2e553fdc52f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:37.971119 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dab1b0-8070-453a-a15e-b2e553fdc52f admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:37.983386 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5741d27-2916-4d45-8d74-f9b8907a7659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:37.984122 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5741d27-2916-4d45-8d74-f9b8907a7659 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:37.984465 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5741d27-2916-4d45-8d74-f9b8907a7659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:37.984747 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5741d27-2916-4d45-8d74-f9b8907a7659 admin admin] PATH_INFO: `/v3/projects/413655ef8b2c442e8b78c21ad4c9c64a/users/97db4f949f7941a4b21ae7cfa7875b97/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:38.008704 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5741d27-2916-4d45-8d74-f9b8907a7659 admin admin] RBAC: Authorizing `identity:create_grant(project_id=413655ef8b2c442e8b78c21ad4c9c64a, user_id=97db4f949f7941a4b21ae7cfa7875b97, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:38.011388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5741d27-2916-4d45-8d74-f9b8907a7659 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:38.055752 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1082/2127] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:26:37 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:38.059677 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dab1b0-8070-453a-a15e-b2e553fdc52f admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:38.070432 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dab1b0-8070-453a-a15e-b2e553fdc52f admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:38.075534 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70560ac6-e824-4725-9acb-1f0f1d02ed22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:38.076856 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70560ac6-e824-4725-9acb-1f0f1d02ed22 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:38.077075 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70560ac6-e824-4725-9acb-1f0f1d02ed22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:38.081936 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70560ac6-e824-4725-9acb-1f0f1d02ed22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:38.081936 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70560ac6-e824-4725-9acb-1f0f1d02ed22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:38.081936 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70560ac6-e824-4725-9acb-1f0f1d02ed22 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:38.086883 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1046/2128] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:26:37 2022] PUT /identity/v3/projects/413655ef8b2c442e8b78c21ad4c9c64a/users/97db4f949f7941a4b21ae7cfa7875b97/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:38.095819 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1083/2129] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:26:38.097295 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5741d27-2916-4d45-8d74-f9b8907a7659 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:38.098012 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835d9c1-1dba-4806-9c26-dc3fa1186376 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:38.098382 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835d9c1-1dba-4806-9c26-dc3fa1186376 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:38.098726 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835d9c1-1dba-4806-9c26-dc3fa1186376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:38.129786 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-3835d9c1-1dba-4806-9c26-dc3fa1186376 None None] MFA Rules not processed for user `97db4f949f7941a4b21ae7cfa7875b97`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:38.131201 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:38.131530 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-3835d9c1-1dba-4806-9c26-dc3fa1186376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:38.169934 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5741d27-2916-4d45-8d74-f9b8907a7659 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:38.175399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03624d04-cc01-408e-90f7-b607073517d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:38.176161 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03624d04-cc01-408e-90f7-b607073517d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:38.176574 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03624d04-cc01-408e-90f7-b607073517d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:38.176916 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03624d04-cc01-408e-90f7-b607073517d3 admin admin] PATH_INFO: `/v3/projects/8216d7f7436440419964aebd7f04a3b7/users/d34c8a8603674133b0a180e1454e3e05/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:38.194670 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03624d04-cc01-408e-90f7-b607073517d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8216d7f7436440419964aebd7f04a3b7, user_id=d34c8a8603674133b0a180e1454e3e05, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:38.197554 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03624d04-cc01-408e-90f7-b607073517d3 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:38.203942 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1084/2130] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:38.208444 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835d9c1-1dba-4806-9c26-dc3fa1186376 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:38.219784 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835d9c1-1dba-4806-9c26-dc3fa1186376 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:38.224669 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80614203-6be2-4c68-8ea6-e8b6c5896d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:38.225319 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80614203-6be2-4c68-8ea6-e8b6c5896d0a admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:38.225641 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80614203-6be2-4c68-8ea6-e8b6c5896d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:38.225865 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80614203-6be2-4c68-8ea6-e8b6c5896d0a admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:38.226553 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80614203-6be2-4c68-8ea6-e8b6c5896d0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:38.228478 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80614203-6be2-4c68-8ea6-e8b6c5896d0a admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:38.233037 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:38.330654 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1047/2131] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:26:38 2022] PUT /identity/v3/projects/8216d7f7436440419964aebd7f04a3b7/users/d34c8a8603674133b0a180e1454e3e05/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:38.335398 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03624d04-cc01-408e-90f7-b607073517d3 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:38.367236 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1085/2132] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:26:38 2022] POST /identity/v3/projects => generated 383 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:26:38.371185 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80614203-6be2-4c68-8ea6-e8b6c5896d0a admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:38.402640 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03624d04-cc01-408e-90f7-b607073517d3 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:38.441249 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80614203-6be2-4c68-8ea6-e8b6c5896d0a admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:38.447018 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84428a35-6e8f-4bd4-a9b0-e07e500e7ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:38.447643 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84428a35-6e8f-4bd4-a9b0-e07e500e7ec0 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:38.449138 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84428a35-6e8f-4bd4-a9b0-e07e500e7ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:38.449138 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84428a35-6e8f-4bd4-a9b0-e07e500e7ec0 admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:38.449138 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84428a35-6e8f-4bd4-a9b0-e07e500e7ec0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:38.451465 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84428a35-6e8f-4bd4-a9b0-e07e500e7ec0 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:38.473633 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fcbee8-ae05-4ec7-ba17-95054990c3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:38.474088 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fcbee8-ae05-4ec7-ba17-95054990c3bc admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:38.474355 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fcbee8-ae05-4ec7-ba17-95054990c3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:38.474620 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fcbee8-ae05-4ec7-ba17-95054990c3bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:38.475340 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fcbee8-ae05-4ec7-ba17-95054990c3bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:38.477341 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fcbee8-ae05-4ec7-ba17-95054990c3bc admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:38.487695 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1048/2133] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:26:38.494319 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fcbee8-ae05-4ec7-ba17-95054990c3bc admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:38.504188 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fcbee8-ae05-4ec7-ba17-95054990c3bc admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:38.509542 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd6f874-6287-443e-86d2-f76d7e584a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:38.510179 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6f874-6287-443e-86d2-f76d7e584a8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:38.510457 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6f874-6287-443e-86d2-f76d7e584a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:38.510751 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6f874-6287-443e-86d2-f76d7e584a8b admin admin] PATH_INFO: `/v3/projects/8216d7f7436440419964aebd7f04a3b7/users/d34c8a8603674133b0a180e1454e3e05/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:38.512676 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd6f874-6287-443e-86d2-f76d7e584a8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8216d7f7436440419964aebd7f04a3b7, user_id=d34c8a8603674133b0a180e1454e3e05, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:38.515295 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd6f874-6287-443e-86d2-f76d7e584a8b admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:38.532201 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1049/2134] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:26:38 2022] PUT /identity/v3/projects/8216d7f7436440419964aebd7f04a3b7/users/d34c8a8603674133b0a180e1454e3e05/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:26:38.543503 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14aa50f-8aed-4027-a902-a8d3f4914e7d None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:38.543799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14aa50f-8aed-4027-a902-a8d3f4914e7d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:38.544075 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14aa50f-8aed-4027-a902-a8d3f4914e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:38.560459 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1086/2135] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:26:38 2022] POST /identity/v3/users => generated 298 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:26:38.570174 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84428a35-6e8f-4bd4-a9b0-e07e500e7ec0 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:38.574262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-e14aa50f-8aed-4027-a902-a8d3f4914e7d None None] MFA Rules not processed for user `d34c8a8603674133b0a180e1454e3e05`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:38.575461 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e14aa50f-8aed-4027-a902-a8d3f4914e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:38.579876 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84428a35-6e8f-4bd4-a9b0-e07e500e7ec0 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:38.604967 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:38.644609 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5942e47d-f96e-4a69-a8a2-e11836e3563b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:38.645913 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942e47d-f96e-4a69-a8a2-e11836e3563b admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:38.646337 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942e47d-f96e-4a69-a8a2-e11836e3563b admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:38.646462 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942e47d-f96e-4a69-a8a2-e11836e3563b admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:38.647252 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5942e47d-f96e-4a69-a8a2-e11836e3563b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:38.648831 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5942e47d-f96e-4a69-a8a2-e11836e3563b admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:38.649752 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1050/2136] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:38 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:26:38.661197 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14aa50f-8aed-4027-a902-a8d3f4914e7d None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:38.662580 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e14aa50f-8aed-4027-a902-a8d3f4914e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:38.666688 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1087/2137] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:26:38.671830 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5942e47d-f96e-4a69-a8a2-e11836e3563b admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:38.685869 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5942e47d-f96e-4a69-a8a2-e11836e3563b admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:38.691901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068be5b1-9653-4117-b18b-c9bf16048302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:38.692621 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068be5b1-9653-4117-b18b-c9bf16048302 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:38.693508 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068be5b1-9653-4117-b18b-c9bf16048302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:38.694416 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068be5b1-9653-4117-b18b-c9bf16048302 admin admin] PATH_INFO: `/v3/projects/6dbf417ec8fb46dc8ddec8fe9ac84d93/users/c5f980f66d5443d1af3dd4b939d8bceb/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:38.711143 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068be5b1-9653-4117-b18b-c9bf16048302 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dbf417ec8fb46dc8ddec8fe9ac84d93, user_id=c5f980f66d5443d1af3dd4b939d8bceb, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:38.713881 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068be5b1-9653-4117-b18b-c9bf16048302 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:38.735587 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14aa50f-8aed-4027-a902-a8d3f4914e7d None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:38.739516 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b403d74-8833-48bc-957d-f5a82c7f8a33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:38.740139 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b403d74-8833-48bc-957d-f5a82c7f8a33 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:38.740963 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b403d74-8833-48bc-957d-f5a82c7f8a33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:38.741357 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b403d74-8833-48bc-957d-f5a82c7f8a33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:38.743190 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5b403d74-8833-48bc-957d-f5a82c7f8a33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:38.782202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1088/2138] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:26:38 2022] PUT /identity/v3/projects/6dbf417ec8fb46dc8ddec8fe9ac84d93/users/c5f980f66d5443d1af3dd4b939d8bceb/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:38.787358 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068be5b1-9653-4117-b18b-c9bf16048302 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:38.812763 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b403d74-8833-48bc-957d-f5a82c7f8a33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:38.814525 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b403d74-8833-48bc-957d-f5a82c7f8a33 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:38.824496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1051/2139] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:38 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 14:26:38.836647 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:38.852594 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068be5b1-9653-4117-b18b-c9bf16048302 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:38.857450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f5968-10f6-4beb-a375-5980b88bcacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:38.858280 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f5968-10f6-4beb-a375-5980b88bcacb admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:38.858586 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f5968-10f6-4beb-a375-5980b88bcacb admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:38.858883 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f5968-10f6-4beb-a375-5980b88bcacb admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:38.859593 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f5968-10f6-4beb-a375-5980b88bcacb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:38.861520 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f5968-10f6-4beb-a375-5980b88bcacb admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:38.873864 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1089/2140] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:26:38.878028 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b403d74-8833-48bc-957d-f5a82c7f8a33 service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:38.889588 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b403d74-8833-48bc-957d-f5a82c7f8a33 service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:38.894331 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683515a9-e83e-436d-8aaf-11a30e8bac79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:38.895073 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683515a9-e83e-436d-8aaf-11a30e8bac79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:38.895360 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683515a9-e83e-436d-8aaf-11a30e8bac79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:38.895637 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683515a9-e83e-436d-8aaf-11a30e8bac79 admin admin] PATH_INFO: `/v3/projects/6dbf417ec8fb46dc8ddec8fe9ac84d93/users/c5f980f66d5443d1af3dd4b939d8bceb/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:38.897707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683515a9-e83e-436d-8aaf-11a30e8bac79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dbf417ec8fb46dc8ddec8fe9ac84d93, user_id=c5f980f66d5443d1af3dd4b939d8bceb, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:38.900366 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683515a9-e83e-436d-8aaf-11a30e8bac79 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:38.911859 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1052/2141] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:26:38 2022] PUT /identity/v3/projects/6dbf417ec8fb46dc8ddec8fe9ac84d93/users/c5f980f66d5443d1af3dd4b939d8bceb/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:26:38.922872 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a6cf1-7488-4c53-aa00-55875867f316 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:38.923158 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a6cf1-7488-4c53-aa00-55875867f316 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:38.923433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a6cf1-7488-4c53-aa00-55875867f316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:38.962931 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-1d9a6cf1-7488-4c53-aa00-55875867f316 None None] MFA Rules not processed for user `c5f980f66d5443d1af3dd4b939d8bceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:38.964404 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-1d9a6cf1-7488-4c53-aa00-55875867f316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:39.045257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1090/2142] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:38 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:39.074933 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683515a9-e83e-436d-8aaf-11a30e8bac79 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:39.077488 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-683515a9-e83e-436d-8aaf-11a30e8bac79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:39.145898 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683515a9-e83e-436d-8aaf-11a30e8bac79 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:39.149747 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6f7d5-016a-4e14-973d-099c07444b00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:39.150364 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6f7d5-016a-4e14-973d-099c07444b00 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:39.150642 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6f7d5-016a-4e14-973d-099c07444b00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:39.150914 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6f7d5-016a-4e14-973d-099c07444b00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:39.170188 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:39.209897 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6f7d5-016a-4e14-973d-099c07444b00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:39.214348 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6f7d5-016a-4e14-973d-099c07444b00 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:39.220542 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1053/2143] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:26:41.868960 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a6cf1-7488-4c53-aa00-55875867f316 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:41.878492 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a6cf1-7488-4c53-aa00-55875867f316 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:41.882890 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78d9ea-e29a-4f7d-96e9-290b3b32da80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:41.883489 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78d9ea-e29a-4f7d-96e9-290b3b32da80 service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:41.883764 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78d9ea-e29a-4f7d-96e9-290b3b32da80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:41.884029 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78d9ea-e29a-4f7d-96e9-290b3b32da80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:41.943165 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a78d9ea-e29a-4f7d-96e9-290b3b32da80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:41.945477 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a78d9ea-e29a-4f7d-96e9-290b3b32da80 service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:41.954863 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1091/2144] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:26:43.625685 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6f7d5-016a-4e14-973d-099c07444b00 service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:43.634417 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6f7d5-016a-4e14-973d-099c07444b00 service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:43.638108 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad1ab6f-bfb9-4258-871c-fce6da7788ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:43.638740 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1ab6f-bfb9-4258-871c-fce6da7788ff service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:43.639015 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1ab6f-bfb9-4258-871c-fce6da7788ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:43.639279 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1ab6f-bfb9-4258-871c-fce6da7788ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:43.705860 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad1ab6f-bfb9-4258-871c-fce6da7788ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:43.707439 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad1ab6f-bfb9-4258-871c-fce6da7788ff service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:43.718510 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1054/2145] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:43 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:26:47.999893 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78d9ea-e29a-4f7d-96e9-290b3b32da80 service cinder] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:48.011602 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78d9ea-e29a-4f7d-96e9-290b3b32da80 service cinder] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:48.016185 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc8046-738b-4093-a8ec-773c628b783f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:48.017102 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc8046-738b-4093-a8ec-773c628b783f service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:48.017662 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc8046-738b-4093-a8ec-773c628b783f service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:48.018188 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc8046-738b-4093-a8ec-773c628b783f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:48.035640 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfc8046-738b-4093-a8ec-773c628b783f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:48.040809 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfc8046-738b-4093-a8ec-773c628b783f service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:48.068672 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1092/2146] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:47 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:26:48.438586 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195bb8b8-ff88-4e1d-b635-ad09564e995d None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:48.439095 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195bb8b8-ff88-4e1d-b635-ad09564e995d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:48.439212 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195bb8b8-ff88-4e1d-b635-ad09564e995d None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:48.440565 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1055/2147] 10.222.0.26 () {54 vars in 921 bytes} [Fri Aug 5 14:26:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:26:48.467968 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4953fd-2448-46a1-b3af-e9ad426ce20a None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:48.468532 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4953fd-2448-46a1-b3af-e9ad426ce20a None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:48.468822 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4953fd-2448-46a1-b3af-e9ad426ce20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:48.490687 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-3d4953fd-2448-46a1-b3af-e9ad426ce20a None None] MFA Rules not processed for user `fff761f6f81a46098f94bd2ec2be9c1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:48.491947 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-3d4953fd-2448-46a1-b3af-e9ad426ce20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:48.545940 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:48.575100 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195bb8b8-ff88-4e1d-b635-ad09564e995d None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:48.578204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-195bb8b8-ff88-4e1d-b635-ad09564e995d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:48.581675 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1093/2148] 10.222.0.26 () {58 vars in 1018 bytes} [Fri Aug 5 14:26:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:26:48.598164 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4953fd-2448-46a1-b3af-e9ad426ce20a None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:48.598164 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-3d4953fd-2448-46a1-b3af-e9ad426ce20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:48.612331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4953fd-2448-46a1-b3af-e9ad426ce20a None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:48.618475 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff1f9e0-c616-4507-9149-5428a1529aba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff761f6f81a46098f94bd2ec2be9c1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:48.625275 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff1f9e0-c616-4507-9149-5428a1529aba service placement] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:48.625275 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff1f9e0-c616-4507-9149-5428a1529aba service placement] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:48.625275 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff1f9e0-c616-4507-9149-5428a1529aba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:48.628760 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff1f9e0-c616-4507-9149-5428a1529aba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:48.630207 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff1f9e0-c616-4507-9149-5428a1529aba service placement] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:48.641346 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1094/2149] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 14:26:48.673288 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195bb8b8-ff88-4e1d-b635-ad09564e995d None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:48.678141 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ec597-a7e1-4c95-ba49-9f6e1dbb1ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:48.678778 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ec597-a7e1-4c95-ba49-9f6e1dbb1ff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:48.679061 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ec597-a7e1-4c95-ba49-9f6e1dbb1ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:48.679320 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ec597-a7e1-4c95-ba49-9f6e1dbb1ff3 admin admin] PATH_INFO: `/v3/users/3bbe964ead724a4da59620786b5706d8` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:48.680628 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1ec597-a7e1-4c95-ba49-9f6e1dbb1ff3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bbe964ead724a4da59620786b5706d8)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:48.682522 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1ec597-a7e1-4c95-ba49-9f6e1dbb1ff3 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:48.907145 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-5b1ec597-a7e1-4c95-ba49-9f6e1dbb1ff3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bbe964ead724a4da59620786b5706d8'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:48.913797 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:48.919974 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-5b1ec597-a7e1-4c95-ba49-9f6e1dbb1ff3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bbe964ead724a4da59620786b5706d8'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:49.014779 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-5b1ec597-a7e1-4c95-ba49-9f6e1dbb1ff3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bbe964ead724a4da59620786b5706d8'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:49.015406 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-5b1ec597-a7e1-4c95-ba49-9f6e1dbb1ff3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bbe964ead724a4da59620786b5706d8'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:49.023703 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-5b1ec597-a7e1-4c95-ba49-9f6e1dbb1ff3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bbe964ead724a4da59620786b5706d8'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:49.029780 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1056/2150] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:26:48 2022] DELETE /identity/v3/users/3bbe964ead724a4da59620786b5706d8 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:26:49.297598 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff1f9e0-c616-4507-9149-5428a1529aba service placement] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:49.310024 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff1f9e0-c616-4507-9149-5428a1529aba service placement] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:49.376485 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a07bb8e-6d5b-44eb-9517-f57009410d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:49.377129 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07bb8e-6d5b-44eb-9517-f57009410d64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:49.377408 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07bb8e-6d5b-44eb-9517-f57009410d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:49.377682 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07bb8e-6d5b-44eb-9517-f57009410d64 admin admin] PATH_INFO: `/v3/projects/85f448f6bc5845d1b0567924c78a6162` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:49.378843 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a07bb8e-6d5b-44eb-9517-f57009410d64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85f448f6bc5845d1b0567924c78a6162)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:49.380789 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a07bb8e-6d5b-44eb-9517-f57009410d64 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:49.473588 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9a07bb8e-6d5b-44eb-9517-f57009410d64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85f448f6bc5845d1b0567924c78a6162', 'request_id': 'req-9a07bb8e-6d5b-44eb-9517-f57009410d64'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:49.480667 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9a07bb8e-6d5b-44eb-9517-f57009410d64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85f448f6bc5845d1b0567924c78a6162', 'request_id': 'req-9a07bb8e-6d5b-44eb-9517-f57009410d64'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:49.486402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9a07bb8e-6d5b-44eb-9517-f57009410d64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85f448f6bc5845d1b0567924c78a6162', 'request_id': 'req-9a07bb8e-6d5b-44eb-9517-f57009410d64'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:49.541432 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9a07bb8e-6d5b-44eb-9517-f57009410d64 admin admin] The token cache is being invalidate because project 85f448f6bc5845d1b0567924c78a6162 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:26:49.544653 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9a07bb8e-6d5b-44eb-9517-f57009410d64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:49.544653 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9a07bb8e-6d5b-44eb-9517-f57009410d64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:49.544653 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1095/2151] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:26:49 2022] DELETE /identity/v3/projects/85f448f6bc5845d1b0567924c78a6162 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:49.548796 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4acf7-6e96-41a3-b790-bfc2267986c2 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:49.549258 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4acf7-6e96-41a3-b790-bfc2267986c2 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:49.549362 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4acf7-6e96-41a3-b790-bfc2267986c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:49.570053 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-a8c4acf7-6e96-41a3-b790-bfc2267986c2 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:49.571426 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a8c4acf7-6e96-41a3-b790-bfc2267986c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:49.633914 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1057/2152] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:49.641686 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7b1b0-24c8-46a2-94e2-9b24824d8098 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:49.643413 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7b1b0-24c8-46a2-94e2-9b24824d8098 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:49.643413 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7b1b0-24c8-46a2-94e2-9b24824d8098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:49.661710 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-56e7b1b0-24c8-46a2-94e2-9b24824d8098 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:49.662909 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-56e7b1b0-24c8-46a2-94e2-9b24824d8098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:49.671611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1096/2153] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:26:49.675751 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4acf7-6e96-41a3-b790-bfc2267986c2 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:49.686667 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4acf7-6e96-41a3-b790-bfc2267986c2 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:49.691825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf8fa2-f531-4504-8db2-73cedaf64d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:49.692463 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf8fa2-f531-4504-8db2-73cedaf64d87 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:49.692742 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf8fa2-f531-4504-8db2-73cedaf64d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:49.693014 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf8fa2-f531-4504-8db2-73cedaf64d87 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:49.693752 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf8fa2-f531-4504-8db2-73cedaf64d87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:26:49.694054 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf8fa2-f531-4504-8db2-73cedaf64d87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:49.695695 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf8fa2-f531-4504-8db2-73cedaf64d87 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:49.707446 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1058/2154] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:26:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:26:49.711902 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7b1b0-24c8-46a2-94e2-9b24824d8098 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:49.721277 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7b1b0-24c8-46a2-94e2-9b24824d8098 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:49.726378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da48fa-2276-41af-887b-e74105db5e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:49.726980 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da48fa-2276-41af-887b-e74105db5e47 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:49.727270 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da48fa-2276-41af-887b-e74105db5e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:49.727518 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da48fa-2276-41af-887b-e74105db5e47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:49.728219 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6da48fa-2276-41af-887b-e74105db5e47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:49.730136 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6da48fa-2276-41af-887b-e74105db5e47 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:49.820439 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1097/2155] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:26:49 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:26:49.824571 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf8fa2-f531-4504-8db2-73cedaf64d87 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:49.834419 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf8fa2-f531-4504-8db2-73cedaf64d87 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:49.890873 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66dced9-103e-4674-b6bc-f781820f25e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:49.891486 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66dced9-103e-4674-b6bc-f781820f25e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:49.891763 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66dced9-103e-4674-b6bc-f781820f25e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:49.892036 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66dced9-103e-4674-b6bc-f781820f25e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:49.892744 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66dced9-103e-4674-b6bc-f781820f25e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:49.894648 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66dced9-103e-4674-b6bc-f781820f25e8 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:49.973151 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da48fa-2276-41af-887b-e74105db5e47 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:49.974850 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-a6da48fa-2276-41af-887b-e74105db5e47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:49.988499 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da48fa-2276-41af-887b-e74105db5e47 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:49.993586 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde42408-2cc8-4997-ae26-e74e18dc21d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:49.994209 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde42408-2cc8-4997-ae26-e74e18dc21d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:49.994485 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde42408-2cc8-4997-ae26-e74e18dc21d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:49.994754 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde42408-2cc8-4997-ae26-e74e18dc21d7 admin admin] PATH_INFO: `/v3/users/9322ed807f0b4b6c8f5bd13ad04620a4` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:49.995910 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde42408-2cc8-4997-ae26-e74e18dc21d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9322ed807f0b4b6c8f5bd13ad04620a4)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:49.997824 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde42408-2cc8-4997-ae26-e74e18dc21d7 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:50.017023 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1059/2156] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:26:49 2022] POST /identity/v3/users => generated 287 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:50.021540 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66dced9-103e-4674-b6bc-f781820f25e8 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:50.032003 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66dced9-103e-4674-b6bc-f781820f25e8 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:50.036554 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c4a430-cfa3-4f4e-a33e-5369763fc317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:50.037138 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4a430-cfa3-4f4e-a33e-5369763fc317 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:50.037416 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4a430-cfa3-4f4e-a33e-5369763fc317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:50.037685 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4a430-cfa3-4f4e-a33e-5369763fc317 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:50.039447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c4a430-cfa3-4f4e-a33e-5369763fc317 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:50.041080 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c4a430-cfa3-4f4e-a33e-5369763fc317 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:50.051512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1060/2157] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:26:50.055609 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c4a430-cfa3-4f4e-a33e-5369763fc317 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:50.065204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c4a430-cfa3-4f4e-a33e-5369763fc317 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:50.069836 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f93f0-c9fd-48a2-a867-1199ae07f1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:50.070324 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f93f0-c9fd-48a2-a867-1199ae07f1f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:50.070652 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f93f0-c9fd-48a2-a867-1199ae07f1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:50.070870 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f93f0-c9fd-48a2-a867-1199ae07f1f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:50.071534 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3f93f0-c9fd-48a2-a867-1199ae07f1f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:50.073141 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3f93f0-c9fd-48a2-a867-1199ae07f1f4 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:50.082702 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1061/2158] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:26:50.087671 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f93f0-c9fd-48a2-a867-1199ae07f1f4 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:50.097406 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f93f0-c9fd-48a2-a867-1199ae07f1f4 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:50.103010 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6d2218-e752-4538-b4e4-ab1b3781c9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:50.103688 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d2218-e752-4538-b4e4-ab1b3781c9f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:50.103963 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d2218-e752-4538-b4e4-ab1b3781c9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:50.104235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d2218-e752-4538-b4e4-ab1b3781c9f4 admin admin] PATH_INFO: `/v3/projects/05567514c1b14aa2a7703e8ae3be25e0/users/775b96e827b64beca04b157046bff883/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:50.121198 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6d2218-e752-4538-b4e4-ab1b3781c9f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05567514c1b14aa2a7703e8ae3be25e0, user_id=775b96e827b64beca04b157046bff883, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:50.123919 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6d2218-e752-4538-b4e4-ab1b3781c9f4 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:50.237977 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:50.249903 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-dde42408-2cc8-4997-ae26-e74e18dc21d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9322ed807f0b4b6c8f5bd13ad04620a4'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:50.254007 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1062/2159] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:26:50 2022] PUT /identity/v3/projects/05567514c1b14aa2a7703e8ae3be25e0/users/775b96e827b64beca04b157046bff883/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:26:50.258080 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-dde42408-2cc8-4997-ae26-e74e18dc21d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9322ed807f0b4b6c8f5bd13ad04620a4'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:50.258611 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-dde42408-2cc8-4997-ae26-e74e18dc21d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9322ed807f0b4b6c8f5bd13ad04620a4'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:50.259242 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-dde42408-2cc8-4997-ae26-e74e18dc21d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9322ed807f0b4b6c8f5bd13ad04620a4'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:50.264959 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a63db3-86b2-4fdb-b2d7-7e0382b211cc None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:50.268627 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a63db3-86b2-4fdb-b2d7-7e0382b211cc None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:50.269109 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a63db3-86b2-4fdb-b2d7-7e0382b211cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:50.274553 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-dde42408-2cc8-4997-ae26-e74e18dc21d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9322ed807f0b4b6c8f5bd13ad04620a4'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:50.285470 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:50.300890 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-a6a63db3-86b2-4fdb-b2d7-7e0382b211cc None None] MFA Rules not processed for user `775b96e827b64beca04b157046bff883`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:50.302123 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a6a63db3-86b2-4fdb-b2d7-7e0382b211cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:50.376305 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1098/2160] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:26:49 2022] DELETE /identity/v3/users/9322ed807f0b4b6c8f5bd13ad04620a4 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:26:50.380701 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc33b5eb-50cd-46f6-92c6-97ba6bd86712 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:50.380987 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc33b5eb-50cd-46f6-92c6-97ba6bd86712 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:50.381257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc33b5eb-50cd-46f6-92c6-97ba6bd86712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:50.382683 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1063/2161] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:26:50.393951 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a63db3-86b2-4fdb-b2d7-7e0382b211cc None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:50.395338 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a6a63db3-86b2-4fdb-b2d7-7e0382b211cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:50.402522 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-fc33b5eb-50cd-46f6-92c6-97ba6bd86712 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:50.403690 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-fc33b5eb-50cd-46f6-92c6-97ba6bd86712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:50.462396 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1099/2162] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:26:50.468556 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a63db3-86b2-4fdb-b2d7-7e0382b211cc None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:50.472386 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc33b5eb-50cd-46f6-92c6-97ba6bd86712 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:50.473761 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-fc33b5eb-50cd-46f6-92c6-97ba6bd86712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:50.535736 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1df04f6-1a43-4a4e-9c2f-187604380edd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:50.539185 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df04f6-1a43-4a4e-9c2f-187604380edd service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:50.539185 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df04f6-1a43-4a4e-9c2f-187604380edd service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:50.539185 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df04f6-1a43-4a4e-9c2f-187604380edd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:50.539472 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc33b5eb-50cd-46f6-92c6-97ba6bd86712 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:50.542110 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355bdf71-e5b6-4b9c-85ef-ce810ca0445b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:50.542720 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355bdf71-e5b6-4b9c-85ef-ce810ca0445b service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:50.542998 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355bdf71-e5b6-4b9c-85ef-ce810ca0445b service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:50.543267 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355bdf71-e5b6-4b9c-85ef-ce810ca0445b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:50.552630 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355bdf71-e5b6-4b9c-85ef-ce810ca0445b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:50.554685 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355bdf71-e5b6-4b9c-85ef-ce810ca0445b service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:50.564223 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1100/2163] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 14:26:50.594248 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1df04f6-1a43-4a4e-9c2f-187604380edd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:50.595870 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1df04f6-1a43-4a4e-9c2f-187604380edd service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:50.605450 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1064/2164] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 14:26:50.775965 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:51.018806 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355bdf71-e5b6-4b9c-85ef-ce810ca0445b service neutron] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:51.019018 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-355bdf71-e5b6-4b9c-85ef-ce810ca0445b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:51.053111 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355bdf71-e5b6-4b9c-85ef-ce810ca0445b service neutron] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:51.065071 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0300c2-a4b6-4f6a-bb4b-926cd1cb744d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:51.065071 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0300c2-a4b6-4f6a-bb4b-926cd1cb744d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:51.065071 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0300c2-a4b6-4f6a-bb4b-926cd1cb744d admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:51.065071 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0300c2-a4b6-4f6a-bb4b-926cd1cb744d admin admin] PATH_INFO: `/v3/projects/86da919b98164f8791254e988d13bd80` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:51.065438 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0300c2-a4b6-4f6a-bb4b-926cd1cb744d admin admin] RBAC: Authorizing `identity:delete_project(project_id=86da919b98164f8791254e988d13bd80)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:51.065438 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0300c2-a4b6-4f6a-bb4b-926cd1cb744d admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:51.197531 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:51.198869 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-ba0300c2-a4b6-4f6a-bb4b-926cd1cb744d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86da919b98164f8791254e988d13bd80', 'request_id': 'req-ba0300c2-a4b6-4f6a-bb4b-926cd1cb744d'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:51.204872 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-ba0300c2-a4b6-4f6a-bb4b-926cd1cb744d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86da919b98164f8791254e988d13bd80', 'request_id': 'req-ba0300c2-a4b6-4f6a-bb4b-926cd1cb744d'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:51.211206 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-ba0300c2-a4b6-4f6a-bb4b-926cd1cb744d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86da919b98164f8791254e988d13bd80', 'request_id': 'req-ba0300c2-a4b6-4f6a-bb4b-926cd1cb744d'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:51.323339 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-ba0300c2-a4b6-4f6a-bb4b-926cd1cb744d admin admin] The token cache is being invalidate because project 86da919b98164f8791254e988d13bd80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:26:51.323865 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-ba0300c2-a4b6-4f6a-bb4b-926cd1cb744d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:51.325517 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-ba0300c2-a4b6-4f6a-bb4b-926cd1cb744d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:51.327986 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1101/2165] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:26:51 2022] DELETE /identity/v3/projects/86da919b98164f8791254e988d13bd80 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:51.332639 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1df04f6-1a43-4a4e-9c2f-187604380edd service nova] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:51.354975 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1df04f6-1a43-4a4e-9c2f-187604380edd service nova] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:51.421692 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe148f-923e-4786-bd8d-5510299e76b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:51.422340 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe148f-923e-4786-bd8d-5510299e76b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:51.422618 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe148f-923e-4786-bd8d-5510299e76b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:51.422891 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe148f-923e-4786-bd8d-5510299e76b0 admin admin] PATH_INFO: `/v3/users/229dbb2594e643098e9123a8a13f3019` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:51.424481 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe148f-923e-4786-bd8d-5510299e76b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=229dbb2594e643098e9123a8a13f3019)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:51.426572 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe148f-923e-4786-bd8d-5510299e76b0 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:51.606327 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-b8fe148f-923e-4786-bd8d-5510299e76b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '229dbb2594e643098e9123a8a13f3019'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:51.619693 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-b8fe148f-923e-4786-bd8d-5510299e76b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '229dbb2594e643098e9123a8a13f3019'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:51.693664 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-b8fe148f-923e-4786-bd8d-5510299e76b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '229dbb2594e643098e9123a8a13f3019'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:51.694270 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-b8fe148f-923e-4786-bd8d-5510299e76b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '229dbb2594e643098e9123a8a13f3019'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:51.702433 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-b8fe148f-923e-4786-bd8d-5510299e76b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '229dbb2594e643098e9123a8a13f3019'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:51.708791 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1065/2166] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:26:51 2022] DELETE /identity/v3/users/229dbb2594e643098e9123a8a13f3019 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:26:52.163190 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0300c2-a4b6-4f6a-bb4b-926cd1cb744d admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:52.165320 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-ba0300c2-a4b6-4f6a-bb4b-926cd1cb744d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:52.179189 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0300c2-a4b6-4f6a-bb4b-926cd1cb744d admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:52.237361 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a0694-d959-4c60-a971-3839f814882b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:52.237926 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a0694-d959-4c60-a971-3839f814882b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:52.238202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a0694-d959-4c60-a971-3839f814882b admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:52.238470 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a0694-d959-4c60-a971-3839f814882b admin admin] PATH_INFO: `/v3/projects/b496b1efd25349f3908458c9d212dbcf` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:52.239613 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6a0694-d959-4c60-a971-3839f814882b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b496b1efd25349f3908458c9d212dbcf)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:52.241590 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6a0694-d959-4c60-a971-3839f814882b admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:52.357412 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-cf6a0694-d959-4c60-a971-3839f814882b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b496b1efd25349f3908458c9d212dbcf', 'request_id': 'req-cf6a0694-d959-4c60-a971-3839f814882b'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:52.364087 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-cf6a0694-d959-4c60-a971-3839f814882b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b496b1efd25349f3908458c9d212dbcf', 'request_id': 'req-cf6a0694-d959-4c60-a971-3839f814882b'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:52.370297 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-cf6a0694-d959-4c60-a971-3839f814882b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b496b1efd25349f3908458c9d212dbcf', 'request_id': 'req-cf6a0694-d959-4c60-a971-3839f814882b'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:52.437455 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-cf6a0694-d959-4c60-a971-3839f814882b admin admin] The token cache is being invalidate because project b496b1efd25349f3908458c9d212dbcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:26:52.437745 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-cf6a0694-d959-4c60-a971-3839f814882b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:52.438367 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-cf6a0694-d959-4c60-a971-3839f814882b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:52.439972 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1102/2167] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:26:52 2022] DELETE /identity/v3/projects/b496b1efd25349f3908458c9d212dbcf => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:52.447799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe148f-923e-4786-bd8d-5510299e76b0 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:52.449320 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-b8fe148f-923e-4786-bd8d-5510299e76b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:52.463345 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe148f-923e-4786-bd8d-5510299e76b0 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:52.509219 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:52.521095 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e52038-ad51-4db5-ad6a-dfdb71910836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:52.521773 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e52038-ad51-4db5-ad6a-dfdb71910836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:52.522054 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e52038-ad51-4db5-ad6a-dfdb71910836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:52.522407 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e52038-ad51-4db5-ad6a-dfdb71910836 admin admin] PATH_INFO: `/v3/users/d34c8a8603674133b0a180e1454e3e05` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:52.524301 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e52038-ad51-4db5-ad6a-dfdb71910836 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d34c8a8603674133b0a180e1454e3e05)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:52.526233 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e52038-ad51-4db5-ad6a-dfdb71910836 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:52.736546 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-62e52038-ad51-4db5-ad6a-dfdb71910836 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd34c8a8603674133b0a180e1454e3e05'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:52.748350 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-62e52038-ad51-4db5-ad6a-dfdb71910836 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd34c8a8603674133b0a180e1454e3e05'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:52.824213 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-62e52038-ad51-4db5-ad6a-dfdb71910836 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd34c8a8603674133b0a180e1454e3e05'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:52.824788 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-62e52038-ad51-4db5-ad6a-dfdb71910836 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd34c8a8603674133b0a180e1454e3e05'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:52.831937 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-62e52038-ad51-4db5-ad6a-dfdb71910836 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd34c8a8603674133b0a180e1454e3e05'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:52.838292 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1066/2168] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:26:52 2022] DELETE /identity/v3/users/d34c8a8603674133b0a180e1454e3e05 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:26:53.262318 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a0694-d959-4c60-a971-3839f814882b admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:53.263834 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-cf6a0694-d959-4c60-a971-3839f814882b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:53.277567 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a0694-d959-4c60-a971-3839f814882b admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:53.335761 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af958611-2dba-4bc7-8aab-131cb1ad252c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:53.336360 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af958611-2dba-4bc7-8aab-131cb1ad252c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:53.336671 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af958611-2dba-4bc7-8aab-131cb1ad252c admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:53.336944 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af958611-2dba-4bc7-8aab-131cb1ad252c admin admin] PATH_INFO: `/v3/projects/8216d7f7436440419964aebd7f04a3b7` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:53.338367 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af958611-2dba-4bc7-8aab-131cb1ad252c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8216d7f7436440419964aebd7f04a3b7)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:53.340682 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af958611-2dba-4bc7-8aab-131cb1ad252c admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:53.348642 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:53.450640 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-af958611-2dba-4bc7-8aab-131cb1ad252c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8216d7f7436440419964aebd7f04a3b7', 'request_id': 'req-af958611-2dba-4bc7-8aab-131cb1ad252c'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:53.456705 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-af958611-2dba-4bc7-8aab-131cb1ad252c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8216d7f7436440419964aebd7f04a3b7', 'request_id': 'req-af958611-2dba-4bc7-8aab-131cb1ad252c'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:53.462799 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-af958611-2dba-4bc7-8aab-131cb1ad252c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8216d7f7436440419964aebd7f04a3b7', 'request_id': 'req-af958611-2dba-4bc7-8aab-131cb1ad252c'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:53.557464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-af958611-2dba-4bc7-8aab-131cb1ad252c admin admin] The token cache is being invalidate because project 8216d7f7436440419964aebd7f04a3b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:26:53.557825 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-af958611-2dba-4bc7-8aab-131cb1ad252c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:53.558537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-af958611-2dba-4bc7-8aab-131cb1ad252c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:53.560173 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1103/2169] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:26:53 2022] DELETE /identity/v3/projects/8216d7f7436440419964aebd7f04a3b7 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:53.568098 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fae795-2474-4a90-93b7-b3f7f271cefc None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:53.568475 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fae795-2474-4a90-93b7-b3f7f271cefc None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:53.568907 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fae795-2474-4a90-93b7-b3f7f271cefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:53.589888 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-51fae795-2474-4a90-93b7-b3f7f271cefc None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:53.591050 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-51fae795-2474-4a90-93b7-b3f7f271cefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:53.653985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1067/2170] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:53.661865 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8605386-a554-43e4-bedf-915d8fc9aafb None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:53.662149 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8605386-a554-43e4-bedf-915d8fc9aafb None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:53.662425 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8605386-a554-43e4-bedf-915d8fc9aafb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:53.683356 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-f8605386-a554-43e4-bedf-915d8fc9aafb None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:53.684618 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-f8605386-a554-43e4-bedf-915d8fc9aafb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:53.692893 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1104/2171] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:53.697353 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fae795-2474-4a90-93b7-b3f7f271cefc None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:53.706950 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fae795-2474-4a90-93b7-b3f7f271cefc None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:53.712288 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4243f-8d07-4772-b0a1-ccd9f766d203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:53.712920 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4243f-8d07-4772-b0a1-ccd9f766d203 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:53.713194 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4243f-8d07-4772-b0a1-ccd9f766d203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:53.713463 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4243f-8d07-4772-b0a1-ccd9f766d203 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:53.714149 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af4243f-8d07-4772-b0a1-ccd9f766d203 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:26:53.714446 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af4243f-8d07-4772-b0a1-ccd9f766d203 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:53.716344 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af4243f-8d07-4772-b0a1-ccd9f766d203 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:53.728197 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1068/2172] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:26:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:26:53.732586 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8605386-a554-43e4-bedf-915d8fc9aafb None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:53.741723 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8605386-a554-43e4-bedf-915d8fc9aafb None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:53.746392 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0b1c64-0d54-45c7-b3c7-106fd5edbac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:53.746987 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b1c64-0d54-45c7-b3c7-106fd5edbac3 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:53.747264 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b1c64-0d54-45c7-b3c7-106fd5edbac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:53.747533 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b1c64-0d54-45c7-b3c7-106fd5edbac3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:53.748191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0b1c64-0d54-45c7-b3c7-106fd5edbac3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:53.750264 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0b1c64-0d54-45c7-b3c7-106fd5edbac3 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:53.856909 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1105/2173] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:26:53 2022] POST /identity/v3/projects => generated 375 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:53.861361 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4243f-8d07-4772-b0a1-ccd9f766d203 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:53.872565 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4243f-8d07-4772-b0a1-ccd9f766d203 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:53.924582 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:53.932220 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52537c67-094a-4bbf-a5db-96d1bb094042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:53.935170 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52537c67-094a-4bbf-a5db-96d1bb094042 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:53.935170 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52537c67-094a-4bbf-a5db-96d1bb094042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:53.935170 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52537c67-094a-4bbf-a5db-96d1bb094042 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:53.935170 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52537c67-094a-4bbf-a5db-96d1bb094042 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:53.935966 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52537c67-094a-4bbf-a5db-96d1bb094042 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:54.026100 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1069/2174] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:26:53 2022] POST /identity/v3/users => generated 288 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:54.030265 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0b1c64-0d54-45c7-b3c7-106fd5edbac3 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:54.041313 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0b1c64-0d54-45c7-b3c7-106fd5edbac3 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:54.046279 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64ee126-982d-424e-821a-16e95035873a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:54.046888 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ee126-982d-424e-821a-16e95035873a admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:54.047167 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ee126-982d-424e-821a-16e95035873a admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:54.047683 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ee126-982d-424e-821a-16e95035873a admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:54.048457 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64ee126-982d-424e-821a-16e95035873a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:54.050083 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64ee126-982d-424e-821a-16e95035873a admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:54.062223 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1106/2175] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:26:54.066497 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52537c67-094a-4bbf-a5db-96d1bb094042 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:54.077302 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52537c67-094a-4bbf-a5db-96d1bb094042 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:54.082032 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a66ec32-7c72-4f1f-8451-7b3dfa007c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:54.084665 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66ec32-7c72-4f1f-8451-7b3dfa007c55 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:54.084665 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66ec32-7c72-4f1f-8451-7b3dfa007c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:54.084665 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66ec32-7c72-4f1f-8451-7b3dfa007c55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:54.084665 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a66ec32-7c72-4f1f-8451-7b3dfa007c55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:54.085522 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a66ec32-7c72-4f1f-8451-7b3dfa007c55 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:54.097024 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1070/2176] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:26:54.101904 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64ee126-982d-424e-821a-16e95035873a admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:54.111989 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64ee126-982d-424e-821a-16e95035873a admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:54.117759 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516526b6-999b-4220-ad5d-b211daca7d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:54.118420 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516526b6-999b-4220-ad5d-b211daca7d63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:54.118734 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516526b6-999b-4220-ad5d-b211daca7d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:54.119091 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516526b6-999b-4220-ad5d-b211daca7d63 admin admin] PATH_INFO: `/v3/projects/33905000a2f64751b88bbff67918d252/users/487d7e4081414c2193cb01ff0a202a6b/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:54.133725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516526b6-999b-4220-ad5d-b211daca7d63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33905000a2f64751b88bbff67918d252, user_id=487d7e4081414c2193cb01ff0a202a6b, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:54.136348 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516526b6-999b-4220-ad5d-b211daca7d63 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:54.202013 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1107/2177] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:26:54 2022] PUT /identity/v3/projects/33905000a2f64751b88bbff67918d252/users/487d7e4081414c2193cb01ff0a202a6b/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:26:54.212651 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de305c-4f9e-4e40-a303-546918f3470d None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:54.212935 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de305c-4f9e-4e40-a303-546918f3470d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:54.213341 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de305c-4f9e-4e40-a303-546918f3470d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:54.243044 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-55de305c-4f9e-4e40-a303-546918f3470d None None] MFA Rules not processed for user `487d7e4081414c2193cb01ff0a202a6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:54.244238 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-55de305c-4f9e-4e40-a303-546918f3470d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:54.313589 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1071/2178] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:54.324568 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516526b6-999b-4220-ad5d-b211daca7d63 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:54.326291 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-516526b6-999b-4220-ad5d-b211daca7d63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:54.397143 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516526b6-999b-4220-ad5d-b211daca7d63 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:54.401490 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b17f2d-9bc8-4ee4-a62e-91d421e84216 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:54.402184 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b17f2d-9bc8-4ee4-a62e-91d421e84216 service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:54.402460 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b17f2d-9bc8-4ee4-a62e-91d421e84216 service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:54.402729 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b17f2d-9bc8-4ee4-a62e-91d421e84216 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:54.413099 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b17f2d-9bc8-4ee4-a62e-91d421e84216 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:54.423504 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b17f2d-9bc8-4ee4-a62e-91d421e84216 service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:54.423951 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1108/2179] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:26:54.866816 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de305c-4f9e-4e40-a303-546918f3470d None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:54.868001 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-55de305c-4f9e-4e40-a303-546918f3470d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:54.932234 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de305c-4f9e-4e40-a303-546918f3470d None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:54.938632 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a7b9fd-23cd-4d0e-bd45-55795e780888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:54.939222 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7b9fd-23cd-4d0e-bd45-55795e780888 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:54.939466 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7b9fd-23cd-4d0e-bd45-55795e780888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:54.939735 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7b9fd-23cd-4d0e-bd45-55795e780888 admin admin] PATH_INFO: `/v3/users/775b96e827b64beca04b157046bff883` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:54.940960 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a7b9fd-23cd-4d0e-bd45-55795e780888 admin admin] RBAC: Authorizing `identity:delete_user(user_id=775b96e827b64beca04b157046bff883)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:54.942886 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a7b9fd-23cd-4d0e-bd45-55795e780888 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:55.114791 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-18a7b9fd-23cd-4d0e-bd45-55795e780888 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '775b96e827b64beca04b157046bff883'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:55.129636 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-18a7b9fd-23cd-4d0e-bd45-55795e780888 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '775b96e827b64beca04b157046bff883'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:55.131477 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:55.198381 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-18a7b9fd-23cd-4d0e-bd45-55795e780888 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '775b96e827b64beca04b157046bff883'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:55.199003 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-18a7b9fd-23cd-4d0e-bd45-55795e780888 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '775b96e827b64beca04b157046bff883'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:55.207372 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-18a7b9fd-23cd-4d0e-bd45-55795e780888 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '775b96e827b64beca04b157046bff883'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:55.213781 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1072/2180] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:26:54 2022] DELETE /identity/v3/users/775b96e827b64beca04b157046bff883 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:55.218766 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14179ae0-6d84-46a7-ad06-355c3660fde4 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:55.219058 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14179ae0-6d84-46a7-ad06-355c3660fde4 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:55.219341 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14179ae0-6d84-46a7-ad06-355c3660fde4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:55.238091 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-14179ae0-6d84-46a7-ad06-355c3660fde4 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:55.239304 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-14179ae0-6d84-46a7-ad06-355c3660fde4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:55.269509 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:55.297293 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1109/2181] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:55.307466 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a7b9fd-23cd-4d0e-bd45-55795e780888 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:55.308918 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-18a7b9fd-23cd-4d0e-bd45-55795e780888 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:55.366048 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a7b9fd-23cd-4d0e-bd45-55795e780888 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:55.370241 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e3046-1463-4532-9968-b172d79aa952 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:55.370846 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e3046-1463-4532-9968-b172d79aa952 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:55.371119 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e3046-1463-4532-9968-b172d79aa952 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:55.371385 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e3046-1463-4532-9968-b172d79aa952 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:55.380310 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e3046-1463-4532-9968-b172d79aa952 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:55.381775 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e3046-1463-4532-9968-b172d79aa952 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:55.392673 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1073/2182] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:26:55.805804 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14179ae0-6d84-46a7-ad06-355c3660fde4 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:55.807306 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-14179ae0-6d84-46a7-ad06-355c3660fde4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:55.820614 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14179ae0-6d84-46a7-ad06-355c3660fde4 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:55.825420 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd0cefc-623b-404a-a16d-7aaf4656612d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:55.826167 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0cefc-623b-404a-a16d-7aaf4656612d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:55.826646 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0cefc-623b-404a-a16d-7aaf4656612d admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:55.826930 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0cefc-623b-404a-a16d-7aaf4656612d admin admin] PATH_INFO: `/v3/projects/05567514c1b14aa2a7703e8ae3be25e0` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:55.828195 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd0cefc-623b-404a-a16d-7aaf4656612d admin admin] RBAC: Authorizing `identity:delete_project(project_id=05567514c1b14aa2a7703e8ae3be25e0)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:55.830202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd0cefc-623b-404a-a16d-7aaf4656612d admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:55.929859 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-3fd0cefc-623b-404a-a16d-7aaf4656612d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05567514c1b14aa2a7703e8ae3be25e0', 'request_id': 'req-3fd0cefc-623b-404a-a16d-7aaf4656612d'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:55.936297 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-3fd0cefc-623b-404a-a16d-7aaf4656612d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05567514c1b14aa2a7703e8ae3be25e0', 'request_id': 'req-3fd0cefc-623b-404a-a16d-7aaf4656612d'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:55.942237 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-3fd0cefc-623b-404a-a16d-7aaf4656612d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05567514c1b14aa2a7703e8ae3be25e0', 'request_id': 'req-3fd0cefc-623b-404a-a16d-7aaf4656612d'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:56.031311 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-3fd0cefc-623b-404a-a16d-7aaf4656612d admin admin] The token cache is being invalidate because project 05567514c1b14aa2a7703e8ae3be25e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:26:56.031598 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-3fd0cefc-623b-404a-a16d-7aaf4656612d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:56.032213 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-3fd0cefc-623b-404a-a16d-7aaf4656612d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:56.035025 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1110/2183] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:26:55 2022] DELETE /identity/v3/projects/05567514c1b14aa2a7703e8ae3be25e0 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:56.040006 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a57102-6b98-4bde-a2f4-ee65cdd3baac None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:56.040531 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a57102-6b98-4bde-a2f4-ee65cdd3baac None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:56.040627 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a57102-6b98-4bde-a2f4-ee65cdd3baac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:56.065613 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-38a57102-6b98-4bde-a2f4-ee65cdd3baac None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:56.065613 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-38a57102-6b98-4bde-a2f4-ee65cdd3baac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:56.123519 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1074/2184] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:56.132939 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0839ca-0398-4cea-b4c0-d0b719a63395 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:56.133228 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0839ca-0398-4cea-b4c0-d0b719a63395 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:56.133502 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0839ca-0398-4cea-b4c0-d0b719a63395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:56.159892 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-ca0839ca-0398-4cea-b4c0-d0b719a63395 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:56.161287 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-ca0839ca-0398-4cea-b4c0-d0b719a63395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:56.172033 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1111/2185] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:56.177750 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a57102-6b98-4bde-a2f4-ee65cdd3baac None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:56.189382 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a57102-6b98-4bde-a2f4-ee65cdd3baac None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:56.194971 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d68fb9-48c0-413f-86e6-4b21cbe6b09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:56.196203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d68fb9-48c0-413f-86e6-4b21cbe6b09d admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:56.196559 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d68fb9-48c0-413f-86e6-4b21cbe6b09d admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:56.196826 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d68fb9-48c0-413f-86e6-4b21cbe6b09d admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:56.197605 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d68fb9-48c0-413f-86e6-4b21cbe6b09d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:26:56.197904 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d68fb9-48c0-413f-86e6-4b21cbe6b09d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:56.199493 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d68fb9-48c0-413f-86e6-4b21cbe6b09d admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:56.218423 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0839ca-0398-4cea-b4c0-d0b719a63395 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:56.224104 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1075/2186] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:26:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:26:56.228844 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0839ca-0398-4cea-b4c0-d0b719a63395 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:56.233710 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e980c9-dfce-4ea5-a11a-885b36d1e79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:56.234408 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e980c9-dfce-4ea5-a11a-885b36d1e79d admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:56.234748 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e980c9-dfce-4ea5-a11a-885b36d1e79d admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:56.235083 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e980c9-dfce-4ea5-a11a-885b36d1e79d admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:56.235917 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e980c9-dfce-4ea5-a11a-885b36d1e79d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:56.237929 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e980c9-dfce-4ea5-a11a-885b36d1e79d admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:56.326158 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1112/2187] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:26:56 2022] POST /identity/v3/projects => generated 403 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:26:56.330391 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d68fb9-48c0-413f-86e6-4b21cbe6b09d admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:56.342554 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d68fb9-48c0-413f-86e6-4b21cbe6b09d admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:56.403062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f55d3-f9ab-4e27-8b70-fdf29af564b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:56.403875 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f55d3-f9ab-4e27-8b70-fdf29af564b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:56.404151 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f55d3-f9ab-4e27-8b70-fdf29af564b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:56.404502 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f55d3-f9ab-4e27-8b70-fdf29af564b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:56.405167 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f55d3-f9ab-4e27-8b70-fdf29af564b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:56.407241 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f55d3-f9ab-4e27-8b70-fdf29af564b5 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:56.499042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1076/2188] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:26:56 2022] POST /identity/v3/users => generated 302 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:56.503219 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e980c9-dfce-4ea5-a11a-885b36d1e79d admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:56.514595 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e980c9-dfce-4ea5-a11a-885b36d1e79d admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:56.519348 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73e1bf4-3413-4c1f-857b-228e30693bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:56.519992 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e1bf4-3413-4c1f-857b-228e30693bb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:56.526236 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e1bf4-3413-4c1f-857b-228e30693bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:56.526236 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e1bf4-3413-4c1f-857b-228e30693bb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:56.526236 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73e1bf4-3413-4c1f-857b-228e30693bb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:56.526236 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73e1bf4-3413-4c1f-857b-228e30693bb5 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:56.534032 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1113/2189] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:26:56.538124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f55d3-f9ab-4e27-8b70-fdf29af564b5 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:56.543296 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:56.549815 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f55d3-f9ab-4e27-8b70-fdf29af564b5 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:56.555905 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef95674c-265f-40e0-a9dd-377067a3726a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:56.556561 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95674c-265f-40e0-a9dd-377067a3726a admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:56.557929 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95674c-265f-40e0-a9dd-377067a3726a admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:56.557929 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95674c-265f-40e0-a9dd-377067a3726a admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:56.557929 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef95674c-265f-40e0-a9dd-377067a3726a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:56.559459 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef95674c-265f-40e0-a9dd-377067a3726a admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:56.573305 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1077/2190] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:26:56.576829 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73e1bf4-3413-4c1f-857b-228e30693bb5 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:56.587205 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73e1bf4-3413-4c1f-857b-228e30693bb5 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:56.592458 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93118481-bc5b-4029-b520-0a73eab83f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:56.593131 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93118481-bc5b-4029-b520-0a73eab83f99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:56.593413 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93118481-bc5b-4029-b520-0a73eab83f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:56.593717 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93118481-bc5b-4029-b520-0a73eab83f99 admin admin] PATH_INFO: `/v3/projects/67e4cb512a4b4b3db031641228f7b29d/users/aaf718c6a5e3466d94933b2deb8c43fe/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:56.609430 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93118481-bc5b-4029-b520-0a73eab83f99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67e4cb512a4b4b3db031641228f7b29d, user_id=aaf718c6a5e3466d94933b2deb8c43fe, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:56.612094 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93118481-bc5b-4029-b520-0a73eab83f99 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:56.672665 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1114/2191] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:26:56 2022] PUT /identity/v3/projects/67e4cb512a4b4b3db031641228f7b29d/users/aaf718c6a5e3466d94933b2deb8c43fe/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:26:56.683832 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec16731-d60f-4e8b-a4c2-86a0ad720a01 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:56.684170 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec16731-d60f-4e8b-a4c2-86a0ad720a01 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:56.684467 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec16731-d60f-4e8b-a4c2-86a0ad720a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:56.714415 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-9ec16731-d60f-4e8b-a4c2-86a0ad720a01 None None] MFA Rules not processed for user `aaf718c6a5e3466d94933b2deb8c43fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:56.715891 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-9ec16731-d60f-4e8b-a4c2-86a0ad720a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:56.781378 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1078/2192] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:56 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:56.785939 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93118481-bc5b-4029-b520-0a73eab83f99 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:56.842641 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:56.847479 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93118481-bc5b-4029-b520-0a73eab83f99 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:56.852590 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a5805-4097-4023-bba9-a76d70227729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:56.860673 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a5805-4097-4023-bba9-a76d70227729 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:56.860673 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a5805-4097-4023-bba9-a76d70227729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:56.860673 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a5805-4097-4023-bba9-a76d70227729 admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:56.860673 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a5805-4097-4023-bba9-a76d70227729 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:56.860673 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a5805-4097-4023-bba9-a76d70227729 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:56.918940 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1115/2193] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:26:56 2022] POST /identity/v3/projects => generated 403 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:56.923049 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec16731-d60f-4e8b-a4c2-86a0ad720a01 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:56.934288 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec16731-d60f-4e8b-a4c2-86a0ad720a01 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:56.990901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b812a-3f92-4f23-a605-b26c44e624f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:56.991643 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b812a-3f92-4f23-a605-b26c44e624f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:56.991909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b812a-3f92-4f23-a605-b26c44e624f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:56.992178 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b812a-3f92-4f23-a605-b26c44e624f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:56.992906 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677b812a-3f92-4f23-a605-b26c44e624f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:56.994812 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677b812a-3f92-4f23-a605-b26c44e624f4 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:57.150445 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1079/2194] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:26:56 2022] POST /identity/v3/users => generated 308 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:26:57.153235 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a5805-4097-4023-bba9-a76d70227729 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:57.168188 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a5805-4097-4023-bba9-a76d70227729 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:57.168188 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6999f638-49ea-4587-badc-d9028d4d073b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:57.168479 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999f638-49ea-4587-badc-d9028d4d073b admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:57.168759 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999f638-49ea-4587-badc-d9028d4d073b admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:57.169049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999f638-49ea-4587-badc-d9028d4d073b admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:57.169736 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6999f638-49ea-4587-badc-d9028d4d073b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:57.171296 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6999f638-49ea-4587-badc-d9028d4d073b admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:57.182239 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde4bf8-c7dc-4fb1-84ab-fc017d805809 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:57.182522 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde4bf8-c7dc-4fb1-84ab-fc017d805809 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:57.182794 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde4bf8-c7dc-4fb1-84ab-fc017d805809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:57.186718 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1116/2195] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:26:57.190725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6999f638-49ea-4587-badc-d9028d4d073b admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:57.201449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6999f638-49ea-4587-badc-d9028d4d073b admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:57.202654 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-afde4bf8-c7dc-4fb1-84ab-fc017d805809 None None] MFA Rules not processed for user `487d7e4081414c2193cb01ff0a202a6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:57.203865 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-afde4bf8-c7dc-4fb1-84ab-fc017d805809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:57.206118 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8618a649-4259-404d-aa78-7eeaa5d5b8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:57.206773 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8618a649-4259-404d-aa78-7eeaa5d5b8a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:57.207054 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8618a649-4259-404d-aa78-7eeaa5d5b8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:57.207328 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8618a649-4259-404d-aa78-7eeaa5d5b8a0 admin admin] PATH_INFO: `/v3/projects/2067348f29e84f96a27582d37ffc431d/users/45360a6fb29e48f0ae86f61c819ea580/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:57.222905 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8618a649-4259-404d-aa78-7eeaa5d5b8a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2067348f29e84f96a27582d37ffc431d, user_id=45360a6fb29e48f0ae86f61c819ea580, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:57.225639 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8618a649-4259-404d-aa78-7eeaa5d5b8a0 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:57.260604 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1080/2196] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:26:57.264623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde4bf8-c7dc-4fb1-84ab-fc017d805809 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:57.265958 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-afde4bf8-c7dc-4fb1-84ab-fc017d805809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:57.281110 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde4bf8-c7dc-4fb1-84ab-fc017d805809 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:57.285996 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c596d1-408f-4868-b490-b1340b4c300c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:57.286642 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c596d1-408f-4868-b490-b1340b4c300c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:57.286920 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c596d1-408f-4868-b490-b1340b4c300c admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:57.287191 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c596d1-408f-4868-b490-b1340b4c300c admin admin] PATH_INFO: `/v3/users/97db4f949f7941a4b21ae7cfa7875b97` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:57.288680 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c596d1-408f-4868-b490-b1340b4c300c admin admin] RBAC: Authorizing `identity:delete_user(user_id=97db4f949f7941a4b21ae7cfa7875b97)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:57.290611 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c596d1-408f-4868-b490-b1340b4c300c admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:57.299239 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1117/2197] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:26:57 2022] PUT /identity/v3/projects/2067348f29e84f96a27582d37ffc431d/users/45360a6fb29e48f0ae86f61c819ea580/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:26:57.300170 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8618a649-4259-404d-aa78-7eeaa5d5b8a0 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:57.304520 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:57.368647 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8618a649-4259-404d-aa78-7eeaa5d5b8a0 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:57.373514 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c60dfb1-8a50-48a8-a029-9688dacccd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:57.374049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60dfb1-8a50-48a8-a029-9688dacccd15 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:57.374456 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60dfb1-8a50-48a8-a029-9688dacccd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:57.374727 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60dfb1-8a50-48a8-a029-9688dacccd15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:57.375452 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c60dfb1-8a50-48a8-a029-9688dacccd15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:57.377095 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c60dfb1-8a50-48a8-a029-9688dacccd15 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:57.388388 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1118/2198] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:26:57.394105 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c60dfb1-8a50-48a8-a029-9688dacccd15 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:57.404501 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c60dfb1-8a50-48a8-a029-9688dacccd15 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:57.409436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982458b2-e9e4-49ce-be27-60b84aa8671d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:57.410104 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982458b2-e9e4-49ce-be27-60b84aa8671d admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:57.410390 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982458b2-e9e4-49ce-be27-60b84aa8671d admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:57.410667 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982458b2-e9e4-49ce-be27-60b84aa8671d admin admin] PATH_INFO: `/v3/projects/2067348f29e84f96a27582d37ffc431d/users/45360a6fb29e48f0ae86f61c819ea580/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:57.412639 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982458b2-e9e4-49ce-be27-60b84aa8671d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2067348f29e84f96a27582d37ffc431d, user_id=45360a6fb29e48f0ae86f61c819ea580, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:57.415523 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982458b2-e9e4-49ce-be27-60b84aa8671d admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:57.426979 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1119/2199] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:26:57 2022] PUT /identity/v3/projects/2067348f29e84f96a27582d37ffc431d/users/45360a6fb29e48f0ae86f61c819ea580/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:26:57.432085 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-60c596d1-408f-4868-b490-b1340b4c300c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97db4f949f7941a4b21ae7cfa7875b97'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:57.439532 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a36d5-9017-4dea-8ba8-82138b77fdb1 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:57.439955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a36d5-9017-4dea-8ba8-82138b77fdb1 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:57.440451 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a36d5-9017-4dea-8ba8-82138b77fdb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:57.445812 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-60c596d1-408f-4868-b490-b1340b4c300c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97db4f949f7941a4b21ae7cfa7875b97'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:57.470850 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-de7a36d5-9017-4dea-8ba8-82138b77fdb1 None None] MFA Rules not processed for user `45360a6fb29e48f0ae86f61c819ea580`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:57.472098 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-de7a36d5-9017-4dea-8ba8-82138b77fdb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:57.522226 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-60c596d1-408f-4868-b490-b1340b4c300c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97db4f949f7941a4b21ae7cfa7875b97'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:57.523495 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-60c596d1-408f-4868-b490-b1340b4c300c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97db4f949f7941a4b21ae7cfa7875b97'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:57.534850 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-60c596d1-408f-4868-b490-b1340b4c300c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97db4f949f7941a4b21ae7cfa7875b97'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:57.542987 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1081/2200] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:26:57 2022] DELETE /identity/v3/users/97db4f949f7941a4b21ae7cfa7875b97 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:57.547414 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b80c49-d841-40e8-9023-8280f16c032f None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:57.547793 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b80c49-d841-40e8-9023-8280f16c032f None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:57.548228 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b80c49-d841-40e8-9023-8280f16c032f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:57.554896 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1120/2201] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:57 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:26:57.562451 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a36d5-9017-4dea-8ba8-82138b77fdb1 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:57.563834 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-de7a36d5-9017-4dea-8ba8-82138b77fdb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:57.568655 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-c5b80c49-d841-40e8-9023-8280f16c032f None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:57.570454 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c5b80c49-d841-40e8-9023-8280f16c032f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:57.578179 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:57.640521 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a36d5-9017-4dea-8ba8-82138b77fdb1 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:57.651996 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1082/2202] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:26:57.661369 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b80c49-d841-40e8-9023-8280f16c032f None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:57.662787 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c5b80c49-d841-40e8-9023-8280f16c032f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:57.698991 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c6798-8436-4221-94a2-571d99045118 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:57.699561 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c6798-8436-4221-94a2-571d99045118 service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:57.699862 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c6798-8436-4221-94a2-571d99045118 service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:57.700115 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c6798-8436-4221-94a2-571d99045118 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:57.702001 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-eb9c6798-8436-4221-94a2-571d99045118 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:57.724850 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b80c49-d841-40e8-9023-8280f16c032f None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:57.728991 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3089ebef-7fbc-414f-ad41-290214fa4541 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:57.729605 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089ebef-7fbc-414f-ad41-290214fa4541 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:57.729879 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089ebef-7fbc-414f-ad41-290214fa4541 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:57.730138 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089ebef-7fbc-414f-ad41-290214fa4541 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:57.740643 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3089ebef-7fbc-414f-ad41-290214fa4541 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:57.742511 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3089ebef-7fbc-414f-ad41-290214fa4541 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:57.755630 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1083/2203] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 14:26:57.770470 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c6798-8436-4221-94a2-571d99045118 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:57.772387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c6798-8436-4221-94a2-571d99045118 service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:57.783601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1121/2204] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:26:57 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:26:58.471788 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3089ebef-7fbc-414f-ad41-290214fa4541 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:58.473245 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-3089ebef-7fbc-414f-ad41-290214fa4541 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:58.486879 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3089ebef-7fbc-414f-ad41-290214fa4541 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:58.492177 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc072fd-8b2b-40a4-b3a0-9aaaeb802181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:58.492970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc072fd-8b2b-40a4-b3a0-9aaaeb802181 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:58.493245 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc072fd-8b2b-40a4-b3a0-9aaaeb802181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:58.493513 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc072fd-8b2b-40a4-b3a0-9aaaeb802181 admin admin] PATH_INFO: `/v3/projects/413655ef8b2c442e8b78c21ad4c9c64a` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:58.494698 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc072fd-8b2b-40a4-b3a0-9aaaeb802181 admin admin] RBAC: Authorizing `identity:delete_project(project_id=413655ef8b2c442e8b78c21ad4c9c64a)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:58.502804 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc072fd-8b2b-40a4-b3a0-9aaaeb802181 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:58.644856 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:58.661015 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-0bc072fd-8b2b-40a4-b3a0-9aaaeb802181 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '413655ef8b2c442e8b78c21ad4c9c64a', 'request_id': 'req-0bc072fd-8b2b-40a4-b3a0-9aaaeb802181'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:58.667856 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-0bc072fd-8b2b-40a4-b3a0-9aaaeb802181 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '413655ef8b2c442e8b78c21ad4c9c64a', 'request_id': 'req-0bc072fd-8b2b-40a4-b3a0-9aaaeb802181'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:58.734987 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-0bc072fd-8b2b-40a4-b3a0-9aaaeb802181 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '413655ef8b2c442e8b78c21ad4c9c64a', 'request_id': 'req-0bc072fd-8b2b-40a4-b3a0-9aaaeb802181'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:58.746367 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-0bc072fd-8b2b-40a4-b3a0-9aaaeb802181 admin admin] The token cache is being invalidate because project 413655ef8b2c442e8b78c21ad4c9c64a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:26:58.746664 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-0bc072fd-8b2b-40a4-b3a0-9aaaeb802181 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:58.747273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-0bc072fd-8b2b-40a4-b3a0-9aaaeb802181 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:58.748797 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1084/2205] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:26:58 2022] DELETE /identity/v3/projects/413655ef8b2c442e8b78c21ad4c9c64a => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:58.752908 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c6798-8436-4221-94a2-571d99045118 service cinder] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:58.763591 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c6798-8436-4221-94a2-571d99045118 service cinder] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:58.820854 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80155462-5282-4742-b352-be916077188e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:58.821494 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80155462-5282-4742-b352-be916077188e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:58.821771 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80155462-5282-4742-b352-be916077188e admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:58.822045 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80155462-5282-4742-b352-be916077188e admin admin] PATH_INFO: `/v3/users/c5f980f66d5443d1af3dd4b939d8bceb` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:58.823230 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80155462-5282-4742-b352-be916077188e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5f980f66d5443d1af3dd4b939d8bceb)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:58.825285 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80155462-5282-4742-b352-be916077188e admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:58.848541 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:26:59.046214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-80155462-5282-4742-b352-be916077188e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5f980f66d5443d1af3dd4b939d8bceb'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:59.053146 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-80155462-5282-4742-b352-be916077188e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5f980f66d5443d1af3dd4b939d8bceb'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:59.053973 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-80155462-5282-4742-b352-be916077188e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5f980f66d5443d1af3dd4b939d8bceb'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:59.058931 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-80155462-5282-4742-b352-be916077188e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5f980f66d5443d1af3dd4b939d8bceb'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:59.066209 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-80155462-5282-4742-b352-be916077188e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5f980f66d5443d1af3dd4b939d8bceb'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:59.174620 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1122/2206] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:26:58 2022] DELETE /identity/v3/users/c5f980f66d5443d1af3dd4b939d8bceb => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:59.581704 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc072fd-8b2b-40a4-b3a0-9aaaeb802181 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:26:59.583171 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-0bc072fd-8b2b-40a4-b3a0-9aaaeb802181 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:59.597385 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc072fd-8b2b-40a4-b3a0-9aaaeb802181 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:26:59.656040 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3196fb-9e98-4166-9bfa-857a3ff5e1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:26:59.656701 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3196fb-9e98-4166-9bfa-857a3ff5e1b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:59.656999 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3196fb-9e98-4166-9bfa-857a3ff5e1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:59.657269 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3196fb-9e98-4166-9bfa-857a3ff5e1b1 admin admin] PATH_INFO: `/v3/projects/6dbf417ec8fb46dc8ddec8fe9ac84d93` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:59.658400 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3196fb-9e98-4166-9bfa-857a3ff5e1b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dbf417ec8fb46dc8ddec8fe9ac84d93)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:26:59.660280 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3196fb-9e98-4166-9bfa-857a3ff5e1b1 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:26:59.766461 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ba3196fb-9e98-4166-9bfa-857a3ff5e1b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dbf417ec8fb46dc8ddec8fe9ac84d93', 'request_id': 'req-ba3196fb-9e98-4166-9bfa-857a3ff5e1b1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:59.773424 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ba3196fb-9e98-4166-9bfa-857a3ff5e1b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dbf417ec8fb46dc8ddec8fe9ac84d93', 'request_id': 'req-ba3196fb-9e98-4166-9bfa-857a3ff5e1b1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:59.847025 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ba3196fb-9e98-4166-9bfa-857a3ff5e1b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dbf417ec8fb46dc8ddec8fe9ac84d93', 'request_id': 'req-ba3196fb-9e98-4166-9bfa-857a3ff5e1b1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:59.858409 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ba3196fb-9e98-4166-9bfa-857a3ff5e1b1 admin admin] The token cache is being invalidate because project 6dbf417ec8fb46dc8ddec8fe9ac84d93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:26:59.858698 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ba3196fb-9e98-4166-9bfa-857a3ff5e1b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:59.859349 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ba3196fb-9e98-4166-9bfa-857a3ff5e1b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:26:59.860907 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1085/2207] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:26:59 2022] DELETE /identity/v3/projects/6dbf417ec8fb46dc8ddec8fe9ac84d93 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:26:59.867243 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06b897-f348-40d3-9cc5-1a3a9f4f88c5 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:59.867698 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06b897-f348-40d3-9cc5-1a3a9f4f88c5 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:59.867948 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06b897-f348-40d3-9cc5-1a3a9f4f88c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:59.887814 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-ce06b897-f348-40d3-9cc5-1a3a9f4f88c5 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:59.889038 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-ce06b897-f348-40d3-9cc5-1a3a9f4f88c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:59.948014 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1123/2208] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:59.956279 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddcf1c8-2d7b-4e8b-9a5f-4632bf8d0484 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:26:59.956606 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddcf1c8-2d7b-4e8b-9a5f-4632bf8d0484 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:26:59.956881 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddcf1c8-2d7b-4e8b-9a5f-4632bf8d0484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:26:59.977571 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-4ddcf1c8-2d7b-4e8b-9a5f-4632bf8d0484 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:26:59.978786 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-4ddcf1c8-2d7b-4e8b-9a5f-4632bf8d0484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:26:59.986836 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1086/2209] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:26:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:26:59.991074 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06b897-f348-40d3-9cc5-1a3a9f4f88c5 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:00.001199 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06b897-f348-40d3-9cc5-1a3a9f4f88c5 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:00.005757 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4878be4e-434f-4019-b95d-49d1d028b64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:00.006358 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878be4e-434f-4019-b95d-49d1d028b64c admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:00.006631 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878be4e-434f-4019-b95d-49d1d028b64c admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:00.006900 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878be4e-434f-4019-b95d-49d1d028b64c admin admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:00.008238 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4878be4e-434f-4019-b95d-49d1d028b64c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:27:00.008691 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4878be4e-434f-4019-b95d-49d1d028b64c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:00.010262 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4878be4e-434f-4019-b95d-49d1d028b64c admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:00.021346 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1124/2210] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:26:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:27:00.026340 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddcf1c8-2d7b-4e8b-9a5f-4632bf8d0484 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:00.035821 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddcf1c8-2d7b-4e8b-9a5f-4632bf8d0484 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:00.040563 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137d00a-1b2a-4e66-846e-cf031f00ee65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:00.042336 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137d00a-1b2a-4e66-846e-cf031f00ee65 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:00.042336 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137d00a-1b2a-4e66-846e-cf031f00ee65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:00.042336 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137d00a-1b2a-4e66-846e-cf031f00ee65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:00.042336 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c137d00a-1b2a-4e66-846e-cf031f00ee65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:00.044355 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c137d00a-1b2a-4e66-846e-cf031f00ee65 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:00.147654 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1087/2211] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:27:00 2022] POST /identity/v3/projects => generated 373 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:27:00.153025 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4878be4e-434f-4019-b95d-49d1d028b64c admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:00.163305 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4878be4e-434f-4019-b95d-49d1d028b64c admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:00.219344 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533cd559-78f0-4135-b4bb-6992f78981f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:00.219947 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533cd559-78f0-4135-b4bb-6992f78981f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:00.220223 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533cd559-78f0-4135-b4bb-6992f78981f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:00.220521 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533cd559-78f0-4135-b4bb-6992f78981f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:00.221169 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533cd559-78f0-4135-b4bb-6992f78981f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:00.223054 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533cd559-78f0-4135-b4bb-6992f78981f7 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:00.317656 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1125/2212] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:27:00 2022] POST /identity/v3/users => generated 287 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:27:00.325959 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137d00a-1b2a-4e66-846e-cf031f00ee65 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:00.336177 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137d00a-1b2a-4e66-846e-cf031f00ee65 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:00.340991 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b77db-fb0d-4639-8435-edfa30c5f291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:00.341589 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b77db-fb0d-4639-8435-edfa30c5f291 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:00.341864 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b77db-fb0d-4639-8435-edfa30c5f291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:00.342134 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b77db-fb0d-4639-8435-edfa30c5f291 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:00.342816 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b77db-fb0d-4639-8435-edfa30c5f291 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:00.344398 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b77db-fb0d-4639-8435-edfa30c5f291 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:00.354064 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1088/2213] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:27:00.357900 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533cd559-78f0-4135-b4bb-6992f78981f7 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:00.368750 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533cd559-78f0-4135-b4bb-6992f78981f7 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:00.373444 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad8674d-495f-447c-84a3-ca8cb3790206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:00.374058 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8674d-495f-447c-84a3-ca8cb3790206 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:00.374344 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8674d-495f-447c-84a3-ca8cb3790206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:00.374615 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8674d-495f-447c-84a3-ca8cb3790206 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:00.375328 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad8674d-495f-447c-84a3-ca8cb3790206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:00.376935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad8674d-495f-447c-84a3-ca8cb3790206 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:00.388555 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1126/2214] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:27:00.392524 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b77db-fb0d-4639-8435-edfa30c5f291 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:00.399060 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:27:00.402717 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b77db-fb0d-4639-8435-edfa30c5f291 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:00.407397 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2525dc-9b38-4b4d-9015-110b56c36d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:00.408062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2525dc-9b38-4b4d-9015-110b56c36d88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:00.413206 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2525dc-9b38-4b4d-9015-110b56c36d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:00.413206 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2525dc-9b38-4b4d-9015-110b56c36d88 admin admin] PATH_INFO: `/v3/projects/f935e8a1f82f490d80a05d6c86bd50ca/users/57b8ad00d35f4038bbb73d9112425d6a/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:00.423205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2525dc-9b38-4b4d-9015-110b56c36d88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f935e8a1f82f490d80a05d6c86bd50ca, user_id=57b8ad00d35f4038bbb73d9112425d6a, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:00.425972 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2525dc-9b38-4b4d-9015-110b56c36d88 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:00.524090 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1089/2215] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:27:00 2022] PUT /identity/v3/projects/f935e8a1f82f490d80a05d6c86bd50ca/users/57b8ad00d35f4038bbb73d9112425d6a/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:27:00.535983 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b8fca-2d58-42a0-868b-42261782d56c None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:00.536270 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b8fca-2d58-42a0-868b-42261782d56c None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:00.536587 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b8fca-2d58-42a0-868b-42261782d56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:00.544844 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:27:00.564622 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-476b8fca-2d58-42a0-868b-42261782d56c None None] MFA Rules not processed for user `57b8ad00d35f4038bbb73d9112425d6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:27:00.565837 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-476b8fca-2d58-42a0-868b-42261782d56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:00.630817 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1127/2216] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:27:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:27:00.642491 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2525dc-9b38-4b4d-9015-110b56c36d88 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:00.644486 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-fd2525dc-9b38-4b4d-9015-110b56c36d88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:00.700371 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2525dc-9b38-4b4d-9015-110b56c36d88 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:00.703988 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503b0738-03b3-49ff-b830-5106c1e85ad7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:00.704620 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b0738-03b3-49ff-b830-5106c1e85ad7 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:00.704897 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b0738-03b3-49ff-b830-5106c1e85ad7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:00.705168 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b0738-03b3-49ff-b830-5106c1e85ad7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:00.713657 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503b0738-03b3-49ff-b830-5106c1e85ad7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:00.715432 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503b0738-03b3-49ff-b830-5106c1e85ad7 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:00.724323 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1090/2217] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:27:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:27:00.748020 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b8fca-2d58-42a0-868b-42261782d56c None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:00.749541 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-476b8fca-2d58-42a0-868b-42261782d56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:00.817127 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b8fca-2d58-42a0-868b-42261782d56c None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:00.821923 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32ccb1-3043-498c-b773-532ab58e0a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:00.822574 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32ccb1-3043-498c-b773-532ab58e0a92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:00.822872 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32ccb1-3043-498c-b773-532ab58e0a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:00.823112 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32ccb1-3043-498c-b773-532ab58e0a92 admin admin] PATH_INFO: `/v3/users/aaf718c6a5e3466d94933b2deb8c43fe` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:00.824276 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32ccb1-3043-498c-b773-532ab58e0a92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aaf718c6a5e3466d94933b2deb8c43fe)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:00.826250 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32ccb1-3043-498c-b773-532ab58e0a92 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:01.091980 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-fb32ccb1-3043-498c-b773-532ab58e0a92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aaf718c6a5e3466d94933b2deb8c43fe'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:01.098801 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-fb32ccb1-3043-498c-b773-532ab58e0a92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aaf718c6a5e3466d94933b2deb8c43fe'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:01.099411 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-fb32ccb1-3043-498c-b773-532ab58e0a92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aaf718c6a5e3466d94933b2deb8c43fe'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:01.100019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-fb32ccb1-3043-498c-b773-532ab58e0a92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aaf718c6a5e3466d94933b2deb8c43fe'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:01.113144 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-fb32ccb1-3043-498c-b773-532ab58e0a92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aaf718c6a5e3466d94933b2deb8c43fe'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:01.188228 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1128/2218] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:27:00 2022] DELETE /identity/v3/users/aaf718c6a5e3466d94933b2deb8c43fe => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:27:01.192767 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2fd3d-9a8f-4236-8ef0-89f8ba09e713 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:01.193049 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2fd3d-9a8f-4236-8ef0-89f8ba09e713 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:01.193324 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2fd3d-9a8f-4236-8ef0-89f8ba09e713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:01.213767 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-35c2fd3d-9a8f-4236-8ef0-89f8ba09e713 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:27:01.215215 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-35c2fd3d-9a8f-4236-8ef0-89f8ba09e713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:01.278922 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1091/2219] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:27:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:27:01.290126 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32ccb1-3043-498c-b773-532ab58e0a92 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:01.291574 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-fb32ccb1-3043-498c-b773-532ab58e0a92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:01.305684 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:27:01.352146 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32ccb1-3043-498c-b773-532ab58e0a92 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:01.356236 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9c589-7bc6-415f-9b84-adfbebd8df2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:01.356932 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9c589-7bc6-415f-9b84-adfbebd8df2f service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:01.357212 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9c589-7bc6-415f-9b84-adfbebd8df2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:01.357487 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9c589-7bc6-415f-9b84-adfbebd8df2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:01.368602 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9c589-7bc6-415f-9b84-adfbebd8df2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:01.369321 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9c589-7bc6-415f-9b84-adfbebd8df2f service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:01.379555 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1129/2220] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:27:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:27:01.879255 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2fd3d-9a8f-4236-8ef0-89f8ba09e713 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:01.881044 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-35c2fd3d-9a8f-4236-8ef0-89f8ba09e713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:01.895135 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2fd3d-9a8f-4236-8ef0-89f8ba09e713 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:01.905639 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a665103-a749-438e-a497-9aa70d56a43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:01.906303 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a665103-a749-438e-a497-9aa70d56a43e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:01.906586 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a665103-a749-438e-a497-9aa70d56a43e admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:01.906858 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a665103-a749-438e-a497-9aa70d56a43e admin admin] PATH_INFO: `/v3/projects/67e4cb512a4b4b3db031641228f7b29d` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:01.908029 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a665103-a749-438e-a497-9aa70d56a43e admin admin] RBAC: Authorizing `identity:delete_project(project_id=67e4cb512a4b4b3db031641228f7b29d)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:01.910255 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a665103-a749-438e-a497-9aa70d56a43e admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:02.164741 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-4a665103-a749-438e-a497-9aa70d56a43e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67e4cb512a4b4b3db031641228f7b29d', 'request_id': 'req-4a665103-a749-438e-a497-9aa70d56a43e'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:02.171089 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-4a665103-a749-438e-a497-9aa70d56a43e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67e4cb512a4b4b3db031641228f7b29d', 'request_id': 'req-4a665103-a749-438e-a497-9aa70d56a43e'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:02.220111 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-4a665103-a749-438e-a497-9aa70d56a43e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67e4cb512a4b4b3db031641228f7b29d', 'request_id': 'req-4a665103-a749-438e-a497-9aa70d56a43e'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:02.230986 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-4a665103-a749-438e-a497-9aa70d56a43e admin admin] The token cache is being invalidate because project 67e4cb512a4b4b3db031641228f7b29d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:27:02.231301 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-4a665103-a749-438e-a497-9aa70d56a43e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:02.231928 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-4a665103-a749-438e-a497-9aa70d56a43e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:02.233538 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1092/2221] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:27:01 2022] DELETE /identity/v3/projects/67e4cb512a4b4b3db031641228f7b29d => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:27:02.237607 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9c589-7bc6-415f-9b84-adfbebd8df2f service neutron] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:02.247298 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9c589-7bc6-415f-9b84-adfbebd8df2f service neutron] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:02.302940 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bfcfb4-913b-460a-bd24-9619670e4d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:02.303579 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bfcfb4-913b-460a-bd24-9619670e4d75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:02.303876 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bfcfb4-913b-460a-bd24-9619670e4d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:02.304189 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bfcfb4-913b-460a-bd24-9619670e4d75 admin admin] PATH_INFO: `/v3/users/45360a6fb29e48f0ae86f61c819ea580` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:02.305367 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bfcfb4-913b-460a-bd24-9619670e4d75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45360a6fb29e48f0ae86f61c819ea580)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:02.307305 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bfcfb4-913b-460a-bd24-9619670e4d75 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:02.439696 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-64bfcfb4-913b-460a-bd24-9619670e4d75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45360a6fb29e48f0ae86f61c819ea580'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:02.447757 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-64bfcfb4-913b-460a-bd24-9619670e4d75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45360a6fb29e48f0ae86f61c819ea580'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:02.448663 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-64bfcfb4-913b-460a-bd24-9619670e4d75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45360a6fb29e48f0ae86f61c819ea580'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:02.449384 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-64bfcfb4-913b-460a-bd24-9619670e4d75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45360a6fb29e48f0ae86f61c819ea580'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:02.462816 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-64bfcfb4-913b-460a-bd24-9619670e4d75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45360a6fb29e48f0ae86f61c819ea580'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:02.546403 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1130/2222] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:27:02 2022] DELETE /identity/v3/users/45360a6fb29e48f0ae86f61c819ea580 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:27:03.200360 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a665103-a749-438e-a497-9aa70d56a43e admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:03.201841 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-4a665103-a749-438e-a497-9aa70d56a43e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:03.214629 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a665103-a749-438e-a497-9aa70d56a43e admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:03.220690 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:27:03.280774 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce09e9-3930-4fd4-813f-a4e5a2c9a0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:03.281413 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce09e9-3930-4fd4-813f-a4e5a2c9a0d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:03.281689 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce09e9-3930-4fd4-813f-a4e5a2c9a0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:03.281959 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce09e9-3930-4fd4-813f-a4e5a2c9a0d3 admin admin] PATH_INFO: `/v3/projects/2067348f29e84f96a27582d37ffc431d` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:03.283094 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ce09e9-3930-4fd4-813f-a4e5a2c9a0d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2067348f29e84f96a27582d37ffc431d)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:03.285042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ce09e9-3930-4fd4-813f-a4e5a2c9a0d3 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:03.462065 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-d7ce09e9-3930-4fd4-813f-a4e5a2c9a0d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2067348f29e84f96a27582d37ffc431d', 'request_id': 'req-d7ce09e9-3930-4fd4-813f-a4e5a2c9a0d3'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:03.469875 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-d7ce09e9-3930-4fd4-813f-a4e5a2c9a0d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2067348f29e84f96a27582d37ffc431d', 'request_id': 'req-d7ce09e9-3930-4fd4-813f-a4e5a2c9a0d3'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:03.574645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-d7ce09e9-3930-4fd4-813f-a4e5a2c9a0d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2067348f29e84f96a27582d37ffc431d', 'request_id': 'req-d7ce09e9-3930-4fd4-813f-a4e5a2c9a0d3'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:03.587847 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-d7ce09e9-3930-4fd4-813f-a4e5a2c9a0d3 admin admin] The token cache is being invalidate because project 2067348f29e84f96a27582d37ffc431d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:27:03.588540 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-d7ce09e9-3930-4fd4-813f-a4e5a2c9a0d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:03.589650 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-d7ce09e9-3930-4fd4-813f-a4e5a2c9a0d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:03.595016 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1093/2223] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:27:03 2022] DELETE /identity/v3/projects/2067348f29e84f96a27582d37ffc431d => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:27:03.597206 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cbbc69-3273-4de3-a84c-7820c74e9911 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:03.597695 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cbbc69-3273-4de3-a84c-7820c74e9911 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:03.597894 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cbbc69-3273-4de3-a84c-7820c74e9911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:03.655701 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-c0cbbc69-3273-4de3-a84c-7820c74e9911 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:27:03.656984 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-c0cbbc69-3273-4de3-a84c-7820c74e9911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:03.703457 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:27:03.737278 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1131/2224] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:27:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:27:03.746408 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3cc394-7752-4d40-a0c3-e968a8a64683 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:03.746684 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3cc394-7752-4d40-a0c3-e968a8a64683 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:03.746954 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3cc394-7752-4d40-a0c3-e968a8a64683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:03.768138 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-6a3cc394-7752-4d40-a0c3-e968a8a64683 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:27:03.769813 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-6a3cc394-7752-4d40-a0c3-e968a8a64683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:03.778707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1094/2225] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:27:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:27:03.782902 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cbbc69-3273-4de3-a84c-7820c74e9911 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:03.793050 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cbbc69-3273-4de3-a84c-7820c74e9911 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:03.797872 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c6f67-86ef-47dc-8fb0-0f44c49c1393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:03.798490 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c6f67-86ef-47dc-8fb0-0f44c49c1393 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:03.798942 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c6f67-86ef-47dc-8fb0-0f44c49c1393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:03.799221 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c6f67-86ef-47dc-8fb0-0f44c49c1393 admin admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:03.799923 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5c6f67-86ef-47dc-8fb0-0f44c49c1393 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:27:03.800231 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5c6f67-86ef-47dc-8fb0-0f44c49c1393 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:03.802002 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5c6f67-86ef-47dc-8fb0-0f44c49c1393 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:03.814782 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1132/2226] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:27:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:27:03.819224 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3cc394-7752-4d40-a0c3-e968a8a64683 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:03.829604 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3cc394-7752-4d40-a0c3-e968a8a64683 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:03.835228 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d5799-bde2-468c-879d-95680139e22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:03.837768 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d5799-bde2-468c-879d-95680139e22c admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:03.838047 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d5799-bde2-468c-879d-95680139e22c admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:03.838321 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d5799-bde2-468c-879d-95680139e22c admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:03.840677 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22d5799-bde2-468c-879d-95680139e22c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:03.841306 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22d5799-bde2-468c-879d-95680139e22c admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:03.951324 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1095/2227] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:27:03 2022] POST /identity/v3/projects => generated 371 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:27:03.952596 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c6f67-86ef-47dc-8fb0-0f44c49c1393 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:03.963472 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c6f67-86ef-47dc-8fb0-0f44c49c1393 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:04.019900 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69940d84-0af1-40d1-81af-67d44c4bcde7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:04.020629 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69940d84-0af1-40d1-81af-67d44c4bcde7 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:04.020906 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69940d84-0af1-40d1-81af-67d44c4bcde7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:04.021171 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69940d84-0af1-40d1-81af-67d44c4bcde7 admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:04.021848 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69940d84-0af1-40d1-81af-67d44c4bcde7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:04.024827 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69940d84-0af1-40d1-81af-67d44c4bcde7 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:04.134263 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1133/2228] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:27:03 2022] POST /identity/v3/users => generated 286 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:27:04.138397 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d5799-bde2-468c-879d-95680139e22c admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:04.149434 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d5799-bde2-468c-879d-95680139e22c admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:04.154155 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df36709-5235-4b0c-9e48-af3ddf0b8784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:04.158799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df36709-5235-4b0c-9e48-af3ddf0b8784 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:04.158799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df36709-5235-4b0c-9e48-af3ddf0b8784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:04.158799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df36709-5235-4b0c-9e48-af3ddf0b8784 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:04.158799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df36709-5235-4b0c-9e48-af3ddf0b8784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:04.158799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df36709-5235-4b0c-9e48-af3ddf0b8784 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:04.169603 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1096/2229] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:27:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:27:04.173929 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69940d84-0af1-40d1-81af-67d44c4bcde7 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:04.184928 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69940d84-0af1-40d1-81af-67d44c4bcde7 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:04.189966 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838172f5-4b46-4c3a-97f1-7dea425b1d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:04.190707 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838172f5-4b46-4c3a-97f1-7dea425b1d7a admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:04.191045 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838172f5-4b46-4c3a-97f1-7dea425b1d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:04.191317 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838172f5-4b46-4c3a-97f1-7dea425b1d7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:04.192531 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838172f5-4b46-4c3a-97f1-7dea425b1d7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:04.193794 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838172f5-4b46-4c3a-97f1-7dea425b1d7a admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:04.205344 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1134/2230] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:27:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:27:04.209623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df36709-5235-4b0c-9e48-af3ddf0b8784 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:04.220042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df36709-5235-4b0c-9e48-af3ddf0b8784 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:04.224813 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b925def-e9bb-4390-802f-19c06628ab37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:04.225539 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b925def-e9bb-4390-802f-19c06628ab37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:04.225923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b925def-e9bb-4390-802f-19c06628ab37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:04.226369 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b925def-e9bb-4390-802f-19c06628ab37 admin admin] PATH_INFO: `/v3/projects/74e9d319c5fd41c6a7cbdf35701a1951/users/f66e78381e9a4af39527e0cacc1043a5/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:04.242287 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b925def-e9bb-4390-802f-19c06628ab37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74e9d319c5fd41c6a7cbdf35701a1951, user_id=f66e78381e9a4af39527e0cacc1043a5, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:04.245261 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b925def-e9bb-4390-802f-19c06628ab37 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:04.325421 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1097/2231] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:27:04 2022] PUT /identity/v3/projects/74e9d319c5fd41c6a7cbdf35701a1951/users/f66e78381e9a4af39527e0cacc1043a5/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:27:04.328122 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde985bb-0cca-4782-9ff9-1dace34408de None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:04.328424 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde985bb-0cca-4782-9ff9-1dace34408de None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:04.328720 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde985bb-0cca-4782-9ff9-1dace34408de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:04.350168 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-cde985bb-0cca-4782-9ff9-1dace34408de None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:27:04.357026 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-cde985bb-0cca-4782-9ff9-1dace34408de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:04.434863 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1135/2232] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:27:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:27:04.445364 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b925def-e9bb-4390-802f-19c06628ab37 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:04.447147 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-4b925def-e9bb-4390-802f-19c06628ab37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:04.489131 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:27:04.512046 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b925def-e9bb-4390-802f-19c06628ab37 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:04.516576 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb9484-dc7f-4a85-bb9d-653e7ec9ff57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:04.517203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb9484-dc7f-4a85-bb9d-653e7ec9ff57 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:04.517482 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb9484-dc7f-4a85-bb9d-653e7ec9ff57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:04.517757 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb9484-dc7f-4a85-bb9d-653e7ec9ff57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:04.532481 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afb9484-dc7f-4a85-bb9d-653e7ec9ff57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:04.534291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afb9484-dc7f-4a85-bb9d-653e7ec9ff57 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:04.545595 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1098/2233] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:27:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 14:27:05.997646 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca9fbd-8873-4329-ae57-b66f8bdb3988 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:05.998504 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca9fbd-8873-4329-ae57-b66f8bdb3988 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:05.998932 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca9fbd-8873-4329-ae57-b66f8bdb3988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:06.055500 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-d7ca9fbd-8873-4329-ae57-b66f8bdb3988 None None] MFA Rules not processed for user `f66e78381e9a4af39527e0cacc1043a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:27:06.056768 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-d7ca9fbd-8873-4329-ae57-b66f8bdb3988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:06.092009 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:27:06.128258 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1136/2234] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:27:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:27:06.133739 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb9484-dc7f-4a85-bb9d-653e7ec9ff57 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:06.143422 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb9484-dc7f-4a85-bb9d-653e7ec9ff57 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:06.148456 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba33e4-bc85-4d48-9406-4726a6302ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:06.149192 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba33e4-bc85-4d48-9406-4726a6302ebc admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:06.149472 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba33e4-bc85-4d48-9406-4726a6302ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:06.149740 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba33e4-bc85-4d48-9406-4726a6302ebc admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:06.150395 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba33e4-bc85-4d48-9406-4726a6302ebc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:06.152320 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba33e4-bc85-4d48-9406-4726a6302ebc admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:06.233340 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1099/2235] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:27:06 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:27:06.237323 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ca9fbd-8873-4329-ae57-b66f8bdb3988 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:06.249074 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ca9fbd-8873-4329-ae57-b66f8bdb3988 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:06.305743 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77ed9f-5447-402d-9cf6-30ffee738c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:06.309754 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77ed9f-5447-402d-9cf6-30ffee738c2a admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:06.309754 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77ed9f-5447-402d-9cf6-30ffee738c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:06.309754 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77ed9f-5447-402d-9cf6-30ffee738c2a admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:06.309754 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea77ed9f-5447-402d-9cf6-30ffee738c2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:06.309754 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea77ed9f-5447-402d-9cf6-30ffee738c2a admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:06.415329 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1137/2236] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:27:06 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:27:06.419345 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba33e4-bc85-4d48-9406-4726a6302ebc admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:06.429484 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba33e4-bc85-4d48-9406-4726a6302ebc admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:06.434135 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b628e24-994e-400f-aa12-b2093c955686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:06.434733 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b628e24-994e-400f-aa12-b2093c955686 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:06.435007 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b628e24-994e-400f-aa12-b2093c955686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:06.435278 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b628e24-994e-400f-aa12-b2093c955686 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:06.436351 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b628e24-994e-400f-aa12-b2093c955686 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:06.437620 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b628e24-994e-400f-aa12-b2093c955686 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:06.447716 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1100/2237] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:27:06.451747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77ed9f-5447-402d-9cf6-30ffee738c2a admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:06.462216 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77ed9f-5447-402d-9cf6-30ffee738c2a admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:06.467650 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f10dd0a-ff1a-4bef-8c58-503468707bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:06.467911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10dd0a-ff1a-4bef-8c58-503468707bd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:06.468186 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10dd0a-ff1a-4bef-8c58-503468707bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:06.468486 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10dd0a-ff1a-4bef-8c58-503468707bd2 admin admin] PATH_INFO: `/v3/projects/fd416c88242f4befbdcab6b0b9b4a9b8/users/332cf83af4f14e52a9f40de2c187fd3a/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:06.484206 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f10dd0a-ff1a-4bef-8c58-503468707bd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd416c88242f4befbdcab6b0b9b4a9b8, user_id=332cf83af4f14e52a9f40de2c187fd3a, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:06.486950 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f10dd0a-ff1a-4bef-8c58-503468707bd2 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:06.563496 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1138/2238] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:27:06 2022] PUT /identity/v3/projects/fd416c88242f4befbdcab6b0b9b4a9b8/users/332cf83af4f14e52a9f40de2c187fd3a/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:27:06.567575 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b628e24-994e-400f-aa12-b2093c955686 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:06.578256 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b628e24-994e-400f-aa12-b2093c955686 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:06.635213 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a04a4-4de1-434a-9600-744a917cdbd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:06.635817 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a04a4-4de1-434a-9600-744a917cdbd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:06.636088 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a04a4-4de1-434a-9600-744a917cdbd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:06.636355 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a04a4-4de1-434a-9600-744a917cdbd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:06.637064 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7a04a4-4de1-434a-9600-744a917cdbd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:06.638746 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7a04a4-4de1-434a-9600-744a917cdbd5 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:06.650637 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1101/2239] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:27:06.654844 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f10dd0a-ff1a-4bef-8c58-503468707bd2 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:06.665440 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f10dd0a-ff1a-4bef-8c58-503468707bd2 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:06.670232 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce24510-911d-4a07-93b0-97d8b9967967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:06.670891 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce24510-911d-4a07-93b0-97d8b9967967 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:06.671172 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce24510-911d-4a07-93b0-97d8b9967967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:06.671449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce24510-911d-4a07-93b0-97d8b9967967 admin admin] PATH_INFO: `/v3/projects/fd416c88242f4befbdcab6b0b9b4a9b8/users/332cf83af4f14e52a9f40de2c187fd3a/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:06.673468 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce24510-911d-4a07-93b0-97d8b9967967 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd416c88242f4befbdcab6b0b9b4a9b8, user_id=332cf83af4f14e52a9f40de2c187fd3a, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:06.676220 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce24510-911d-4a07-93b0-97d8b9967967 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:06.687038 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1139/2240] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:27:06 2022] PUT /identity/v3/projects/fd416c88242f4befbdcab6b0b9b4a9b8/users/332cf83af4f14e52a9f40de2c187fd3a/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:27:08.116363 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af80efd-236f-4d7b-809d-87ead45a65cb None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:08.117157 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af80efd-236f-4d7b-809d-87ead45a65cb None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:08.117519 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af80efd-236f-4d7b-809d-87ead45a65cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:08.162788 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-9af80efd-236f-4d7b-809d-87ead45a65cb None None] MFA Rules not processed for user `332cf83af4f14e52a9f40de2c187fd3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:27:08.164248 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-9af80efd-236f-4d7b-809d-87ead45a65cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:08.247563 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1102/2241] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:27:08 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:27:08.261831 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce24510-911d-4a07-93b0-97d8b9967967 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:08.263867 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-6ce24510-911d-4a07-93b0-97d8b9967967 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:08.355402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce24510-911d-4a07-93b0-97d8b9967967 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:08.359371 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791c61b-9f1a-4b2e-a661-eb1513a5e592 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:08.360127 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791c61b-9f1a-4b2e-a661-eb1513a5e592 service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:08.360546 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791c61b-9f1a-4b2e-a661-eb1513a5e592 service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:08.360925 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791c61b-9f1a-4b2e-a661-eb1513a5e592 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:08.414737 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:27:08.437358 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3791c61b-9f1a-4b2e-a661-eb1513a5e592 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:08.438867 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3791c61b-9f1a-4b2e-a661-eb1513a5e592 service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:08.466812 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1140/2242] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:27:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:27:10.382285 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af80efd-236f-4d7b-809d-87ead45a65cb None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:10.384340 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-9af80efd-236f-4d7b-809d-87ead45a65cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:10.398079 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af80efd-236f-4d7b-809d-87ead45a65cb None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:10.407365 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f944f38-d360-47e4-8081-d0510db843eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:10.408202 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f944f38-d360-47e4-8081-d0510db843eb service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:10.408693 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f944f38-d360-47e4-8081-d0510db843eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:10.409145 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f944f38-d360-47e4-8081-d0510db843eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:10.420627 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f944f38-d360-47e4-8081-d0510db843eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:10.422425 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f944f38-d360-47e4-8081-d0510db843eb service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:10.448064 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1103/2243] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:27:10 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:27:19.546470 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3a5a2-44e7-445e-914a-353098000587 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:19.547046 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3a5a2-44e7-445e-914a-353098000587 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:19.547046 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3a5a2-44e7-445e-914a-353098000587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:19.567570 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-1dc3a5a2-44e7-445e-914a-353098000587 None None] MFA Rules not processed for user `b9d62fe4e6de4308a3a51be3e900625f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:27:19.568811 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-1dc3a5a2-44e7-445e-914a-353098000587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:19.624757 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1141/2244] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:27:19 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:27:19.723233 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f944f38-d360-47e4-8081-d0510db843eb service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:19.727237 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-3f944f38-d360-47e4-8081-d0510db843eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:19.744988 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:27:19.799068 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f944f38-d360-47e4-8081-d0510db843eb service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:19.804721 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f749e-88e6-443d-9f9d-df6f119f8d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:19.805368 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f749e-88e6-443d-9f9d-df6f119f8d3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:19.805648 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f749e-88e6-443d-9f9d-df6f119f8d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:19.806012 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f749e-88e6-443d-9f9d-df6f119f8d3f admin admin] PATH_INFO: `/v3/users/57b8ad00d35f4038bbb73d9112425d6a` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:19.807922 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33f749e-88e6-443d-9f9d-df6f119f8d3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=57b8ad00d35f4038bbb73d9112425d6a)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:19.809880 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33f749e-88e6-443d-9f9d-df6f119f8d3f admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:20.017320 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f33f749e-88e6-443d-9f9d-df6f119f8d3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57b8ad00d35f4038bbb73d9112425d6a'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:20.035838 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f33f749e-88e6-443d-9f9d-df6f119f8d3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57b8ad00d35f4038bbb73d9112425d6a'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:20.197806 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f33f749e-88e6-443d-9f9d-df6f119f8d3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57b8ad00d35f4038bbb73d9112425d6a'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:20.200690 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f33f749e-88e6-443d-9f9d-df6f119f8d3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57b8ad00d35f4038bbb73d9112425d6a'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:20.207360 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f33f749e-88e6-443d-9f9d-df6f119f8d3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57b8ad00d35f4038bbb73d9112425d6a'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:20.218876 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1104/2245] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:27:19 2022] DELETE /identity/v3/users/57b8ad00d35f4038bbb73d9112425d6a => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:27:20.224092 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e6641-cb18-4232-b987-a8d444f25160 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:20.224092 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e6641-cb18-4232-b987-a8d444f25160 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:20.224092 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e6641-cb18-4232-b987-a8d444f25160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:20.244632 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-724e6641-cb18-4232-b987-a8d444f25160 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:27:20.245866 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-724e6641-cb18-4232-b987-a8d444f25160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:20.326192 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1142/2246] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:27:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:27:20.336114 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f749e-88e6-443d-9f9d-df6f119f8d3f admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:20.337522 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f33f749e-88e6-443d-9f9d-df6f119f8d3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:20.430622 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f749e-88e6-443d-9f9d-df6f119f8d3f admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:20.435994 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06276a62-3e94-4759-ba44-9f37f716b16c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:20.436850 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06276a62-3e94-4759-ba44-9f37f716b16c service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:20.437517 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06276a62-3e94-4759-ba44-9f37f716b16c service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:20.437793 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06276a62-3e94-4759-ba44-9f37f716b16c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:20.451780 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06276a62-3e94-4759-ba44-9f37f716b16c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:20.453281 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06276a62-3e94-4759-ba44-9f37f716b16c service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:20.464244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1105/2247] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:27:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 14:27:21.021499 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e6641-cb18-4232-b987-a8d444f25160 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:21.022485 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-724e6641-cb18-4232-b987-a8d444f25160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:21.036856 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e6641-cb18-4232-b987-a8d444f25160 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:21.044228 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9545f7e-707e-44cd-bb14-918423f763d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:21.044960 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9545f7e-707e-44cd-bb14-918423f763d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:21.045994 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9545f7e-707e-44cd-bb14-918423f763d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:21.047466 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9545f7e-707e-44cd-bb14-918423f763d3 admin admin] PATH_INFO: `/v3/projects/f935e8a1f82f490d80a05d6c86bd50ca` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:21.049570 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9545f7e-707e-44cd-bb14-918423f763d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f935e8a1f82f490d80a05d6c86bd50ca)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:21.051954 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9545f7e-707e-44cd-bb14-918423f763d3 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:21.209613 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-b9545f7e-707e-44cd-bb14-918423f763d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f935e8a1f82f490d80a05d6c86bd50ca', 'request_id': 'req-b9545f7e-707e-44cd-bb14-918423f763d3'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:21.220975 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-b9545f7e-707e-44cd-bb14-918423f763d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f935e8a1f82f490d80a05d6c86bd50ca', 'request_id': 'req-b9545f7e-707e-44cd-bb14-918423f763d3'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:21.230038 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-b9545f7e-707e-44cd-bb14-918423f763d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f935e8a1f82f490d80a05d6c86bd50ca', 'request_id': 'req-b9545f7e-707e-44cd-bb14-918423f763d3'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:21.315883 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-b9545f7e-707e-44cd-bb14-918423f763d3 admin admin] The token cache is being invalidate because project f935e8a1f82f490d80a05d6c86bd50ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:27:21.318209 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-b9545f7e-707e-44cd-bb14-918423f763d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:21.318209 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-b9545f7e-707e-44cd-bb14-918423f763d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:21.318476 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1143/2248] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:27:21 2022] DELETE /identity/v3/projects/f935e8a1f82f490d80a05d6c86bd50ca => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:27:21.323804 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2203e34-99be-4ff6-9b51-6d1e3664526d None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:21.324314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2203e34-99be-4ff6-9b51-6d1e3664526d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:21.324442 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2203e34-99be-4ff6-9b51-6d1e3664526d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:21.344281 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-d2203e34-99be-4ff6-9b51-6d1e3664526d None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:27:21.347078 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d2203e34-99be-4ff6-9b51-6d1e3664526d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:21.376944 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:27:21.407817 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1106/2249] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:27:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:27:21.416949 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec62daf-11c0-4ccb-a93d-902cb5aaea76 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:21.417267 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec62daf-11c0-4ccb-a93d-902cb5aaea76 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:21.417547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec62daf-11c0-4ccb-a93d-902cb5aaea76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:21.432547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:27:21.443387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-5ec62daf-11c0-4ccb-a93d-902cb5aaea76 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:27:21.443387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-5ec62daf-11c0-4ccb-a93d-902cb5aaea76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:21.448045 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1144/2250] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:27:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:27:21.452286 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2203e34-99be-4ff6-9b51-6d1e3664526d None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:21.462397 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2203e34-99be-4ff6-9b51-6d1e3664526d None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:21.468740 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45520a0-5800-42e4-8d37-049b0793e86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:21.469400 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45520a0-5800-42e4-8d37-049b0793e86c admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:21.473288 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45520a0-5800-42e4-8d37-049b0793e86c admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:21.473288 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45520a0-5800-42e4-8d37-049b0793e86c admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:21.473288 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45520a0-5800-42e4-8d37-049b0793e86c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:27:21.473288 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45520a0-5800-42e4-8d37-049b0793e86c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:21.473288 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45520a0-5800-42e4-8d37-049b0793e86c admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:21.485988 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1107/2251] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:27:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:27:21.490390 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec62daf-11c0-4ccb-a93d-902cb5aaea76 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:21.501139 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec62daf-11c0-4ccb-a93d-902cb5aaea76 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:21.505838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f4aa0-8d75-4c95-90cb-e0d91fe11f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:21.506489 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f4aa0-8d75-4c95-90cb-e0d91fe11f7c admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:21.506796 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f4aa0-8d75-4c95-90cb-e0d91fe11f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:21.507065 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f4aa0-8d75-4c95-90cb-e0d91fe11f7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:21.507724 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11f4aa0-8d75-4c95-90cb-e0d91fe11f7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:21.510956 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11f4aa0-8d75-4c95-90cb-e0d91fe11f7c admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:21.616904 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1145/2252] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:27:21 2022] POST /identity/v3/projects => generated 391 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:27:21.617789 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45520a0-5800-42e4-8d37-049b0793e86c admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:21.628399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45520a0-5800-42e4-8d37-049b0793e86c admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:21.692615 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615537e-1ddb-48fc-8a82-7df38a6b9a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:21.693236 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615537e-1ddb-48fc-8a82-7df38a6b9a71 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:21.693539 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615537e-1ddb-48fc-8a82-7df38a6b9a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:21.693982 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615537e-1ddb-48fc-8a82-7df38a6b9a71 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:21.694759 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2615537e-1ddb-48fc-8a82-7df38a6b9a71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:21.696812 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2615537e-1ddb-48fc-8a82-7df38a6b9a71 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:21.802557 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1108/2253] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:27:21 2022] POST /identity/v3/users => generated 296 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:27:21.806956 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f4aa0-8d75-4c95-90cb-e0d91fe11f7c admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:21.817875 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f4aa0-8d75-4c95-90cb-e0d91fe11f7c admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:21.823166 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd1a3e5-75fa-4dac-98b8-23711bc78135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:21.823876 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1a3e5-75fa-4dac-98b8-23711bc78135 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:21.824158 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1a3e5-75fa-4dac-98b8-23711bc78135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:21.824463 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1a3e5-75fa-4dac-98b8-23711bc78135 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:21.825162 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd1a3e5-75fa-4dac-98b8-23711bc78135 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:21.828740 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd1a3e5-75fa-4dac-98b8-23711bc78135 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:21.836996 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1146/2254] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:27:21.841895 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615537e-1ddb-48fc-8a82-7df38a6b9a71 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:21.853931 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615537e-1ddb-48fc-8a82-7df38a6b9a71 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:21.859305 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae985bb-8021-49db-bbf2-68acfff525f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:21.859496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae985bb-8021-49db-bbf2-68acfff525f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:21.859742 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae985bb-8021-49db-bbf2-68acfff525f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:21.860016 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae985bb-8021-49db-bbf2-68acfff525f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:21.860803 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae985bb-8021-49db-bbf2-68acfff525f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:21.862385 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae985bb-8021-49db-bbf2-68acfff525f6 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:21.873731 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1109/2255] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:27:21.877878 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd1a3e5-75fa-4dac-98b8-23711bc78135 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:21.889111 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd1a3e5-75fa-4dac-98b8-23711bc78135 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:21.894537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cfe5b7-d2cc-42d0-b6f2-94bcb7e9a62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:21.895073 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfe5b7-d2cc-42d0-b6f2-94bcb7e9a62a admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:21.895356 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfe5b7-d2cc-42d0-b6f2-94bcb7e9a62a admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:21.895629 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfe5b7-d2cc-42d0-b6f2-94bcb7e9a62a admin admin] PATH_INFO: `/v3/projects/adcf03f157f24ddcb558a243b6174c54/users/ab9d7613f0124496a0fa959347998ced/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:21.912594 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cfe5b7-d2cc-42d0-b6f2-94bcb7e9a62a admin admin] RBAC: Authorizing `identity:create_grant(project_id=adcf03f157f24ddcb558a243b6174c54, user_id=ab9d7613f0124496a0fa959347998ced, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:21.915327 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cfe5b7-d2cc-42d0-b6f2-94bcb7e9a62a admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:21.992111 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1147/2256] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:27:21 2022] PUT /identity/v3/projects/adcf03f157f24ddcb558a243b6174c54/users/ab9d7613f0124496a0fa959347998ced/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:27:22.003514 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93c2f0-8bfc-4323-824e-1de1dc248524 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:22.003795 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93c2f0-8bfc-4323-824e-1de1dc248524 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:22.004138 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93c2f0-8bfc-4323-824e-1de1dc248524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:22.034944 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-8c93c2f0-8bfc-4323-824e-1de1dc248524 None None] MFA Rules not processed for user `ab9d7613f0124496a0fa959347998ced`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:27:22.036193 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8c93c2f0-8bfc-4323-824e-1de1dc248524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:22.103516 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1110/2257] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:27:22 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:27:22.114566 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cfe5b7-d2cc-42d0-b6f2-94bcb7e9a62a admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:22.116266 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-14cfe5b7-d2cc-42d0-b6f2-94bcb7e9a62a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:22.174120 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cfe5b7-d2cc-42d0-b6f2-94bcb7e9a62a admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:22.177747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5fd8a-3e01-4cdd-859c-9bfa4957470c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:22.178404 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5fd8a-3e01-4cdd-859c-9bfa4957470c service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:22.178684 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5fd8a-3e01-4cdd-859c-9bfa4957470c service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:22.178960 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5fd8a-3e01-4cdd-859c-9bfa4957470c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:22.188426 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5fd8a-3e01-4cdd-859c-9bfa4957470c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:22.189995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5fd8a-3e01-4cdd-859c-9bfa4957470c service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:22.200139 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1148/2258] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:27:22 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:27:25.281621 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93c2f0-8bfc-4323-824e-1de1dc248524 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:25.283118 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8c93c2f0-8bfc-4323-824e-1de1dc248524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:25.315533 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:27:25.349840 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93c2f0-8bfc-4323-824e-1de1dc248524 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:25.354401 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e34bc79-a941-4b63-9837-5dfd17cfd172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:25.358698 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34bc79-a941-4b63-9837-5dfd17cfd172 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:25.358698 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34bc79-a941-4b63-9837-5dfd17cfd172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:25.358698 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34bc79-a941-4b63-9837-5dfd17cfd172 admin admin] PATH_INFO: `/v3/users/d4251382d5d34b5f8fc6037f966f84ee` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:25.358698 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e34bc79-a941-4b63-9837-5dfd17cfd172 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4251382d5d34b5f8fc6037f966f84ee)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:25.358698 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e34bc79-a941-4b63-9837-5dfd17cfd172 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:25.644769 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8e34bc79-a941-4b63-9837-5dfd17cfd172 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4251382d5d34b5f8fc6037f966f84ee'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:25.662159 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8e34bc79-a941-4b63-9837-5dfd17cfd172 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4251382d5d34b5f8fc6037f966f84ee'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:25.739400 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8e34bc79-a941-4b63-9837-5dfd17cfd172 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4251382d5d34b5f8fc6037f966f84ee'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:25.740217 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8e34bc79-a941-4b63-9837-5dfd17cfd172 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4251382d5d34b5f8fc6037f966f84ee'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:25.748435 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8e34bc79-a941-4b63-9837-5dfd17cfd172 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4251382d5d34b5f8fc6037f966f84ee'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:25.755015 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1111/2259] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:27:25 2022] DELETE /identity/v3/users/d4251382d5d34b5f8fc6037f966f84ee => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:27:25.759562 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c1d80-eead-4148-8fde-f49a6f3ceed9 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:25.760023 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c1d80-eead-4148-8fde-f49a6f3ceed9 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:25.760102 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c1d80-eead-4148-8fde-f49a6f3ceed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:25.782075 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-d15c1d80-eead-4148-8fde-f49a6f3ceed9 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:27:25.783286 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-d15c1d80-eead-4148-8fde-f49a6f3ceed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:25.849113 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1149/2260] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:27:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:27:25.860254 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e34bc79-a941-4b63-9837-5dfd17cfd172 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:25.861928 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8e34bc79-a941-4b63-9837-5dfd17cfd172 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:25.933544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e34bc79-a941-4b63-9837-5dfd17cfd172 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:25.937386 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2134ca-4849-476d-a875-72ca7c4d9620 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:25.938028 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2134ca-4849-476d-a875-72ca7c4d9620 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:25.938323 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2134ca-4849-476d-a875-72ca7c4d9620 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:25.938552 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2134ca-4849-476d-a875-72ca7c4d9620 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:25.950632 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2134ca-4849-476d-a875-72ca7c4d9620 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:25.952073 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2134ca-4849-476d-a875-72ca7c4d9620 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:25.983824 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1112/2261] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:27:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:27:26.364116 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8742875-15f4-4bdf-be05-c60449d9f646 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:26.364116 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8742875-15f4-4bdf-be05-c60449d9f646 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:26.364116 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8742875-15f4-4bdf-be05-c60449d9f646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:26.383509 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-d8742875-15f4-4bdf-be05-c60449d9f646 None None] MFA Rules not processed for user `17e3b509a2de4159bf234594573f5e18`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:27:26.384806 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-d8742875-15f4-4bdf-be05-c60449d9f646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:26.430604 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:27:26.445953 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1150/2262] 10.222.0.26 () {58 vars in 1009 bytes} [Fri Aug 5 14:27:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:27:26.496025 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2134ca-4849-476d-a875-72ca7c4d9620 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:26.497544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-bd2134ca-4849-476d-a875-72ca7c4d9620 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:26.549329 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:27:26.566228 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2134ca-4849-476d-a875-72ca7c4d9620 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:26.571185 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e79881b-b5ec-463e-8f67-886c0510b40b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dddd7eea67f34f0cbb5f6a1cd85f1fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:26.572303 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79881b-b5ec-463e-8f67-886c0510b40b service swift] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:26.572303 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79881b-b5ec-463e-8f67-886c0510b40b service swift] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:26.572534 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79881b-b5ec-463e-8f67-886c0510b40b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:26.580887 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e79881b-b5ec-463e-8f67-886c0510b40b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:26.582464 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e79881b-b5ec-463e-8f67-886c0510b40b service swift] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:26.591506 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1113/2263] 10.222.0.26 () {60 vars in 1384 bytes} [Fri Aug 5 14:27:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 05 14:27:26.613305 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8742875-15f4-4bdf-be05-c60449d9f646 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:26.614796 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-d8742875-15f4-4bdf-be05-c60449d9f646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:26.635969 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8742875-15f4-4bdf-be05-c60449d9f646 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:26.640804 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00865a18-2889-4b38-80ef-08b18bd87a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:26.641451 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00865a18-2889-4b38-80ef-08b18bd87a1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:26.641755 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00865a18-2889-4b38-80ef-08b18bd87a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:26.642002 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00865a18-2889-4b38-80ef-08b18bd87a1a admin admin] PATH_INFO: `/v3/projects/580720a3054d477986b3d1feb1f99781` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:26.649251 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00865a18-2889-4b38-80ef-08b18bd87a1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=580720a3054d477986b3d1feb1f99781)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:26.652288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00865a18-2889-4b38-80ef-08b18bd87a1a admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:26.782964 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-00865a18-2889-4b38-80ef-08b18bd87a1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '580720a3054d477986b3d1feb1f99781', 'request_id': 'req-00865a18-2889-4b38-80ef-08b18bd87a1a'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:26.789427 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-00865a18-2889-4b38-80ef-08b18bd87a1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '580720a3054d477986b3d1feb1f99781', 'request_id': 'req-00865a18-2889-4b38-80ef-08b18bd87a1a'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:26.796301 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-00865a18-2889-4b38-80ef-08b18bd87a1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '580720a3054d477986b3d1feb1f99781', 'request_id': 'req-00865a18-2889-4b38-80ef-08b18bd87a1a'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:26.886012 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-00865a18-2889-4b38-80ef-08b18bd87a1a admin admin] The token cache is being invalidate because project 580720a3054d477986b3d1feb1f99781 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:27:26.887249 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-00865a18-2889-4b38-80ef-08b18bd87a1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:26.887249 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-00865a18-2889-4b38-80ef-08b18bd87a1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:26.888856 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1151/2264] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:27:26 2022] DELETE /identity/v3/projects/580720a3054d477986b3d1feb1f99781 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:27:26.893719 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f19c93-80f8-42a8-99c1-f8960ef46f00 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:26.894201 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f19c93-80f8-42a8-99c1-f8960ef46f00 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:26.894477 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f19c93-80f8-42a8-99c1-f8960ef46f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:26.916831 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-03f19c93-80f8-42a8-99c1-f8960ef46f00 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:27:26.918585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-03f19c93-80f8-42a8-99c1-f8960ef46f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:26.980488 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1114/2265] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:27:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:27:26.988541 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583c78d-476a-45f8-b42d-8b3c88fba8df None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:26.988873 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583c78d-476a-45f8-b42d-8b3c88fba8df None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:26.989446 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583c78d-476a-45f8-b42d-8b3c88fba8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:27.008725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-6583c78d-476a-45f8-b42d-8b3c88fba8df None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:27:27.010553 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-6583c78d-476a-45f8-b42d-8b3c88fba8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:27.018411 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1152/2266] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:27:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:27:27.022612 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f19c93-80f8-42a8-99c1-f8960ef46f00 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:27.032718 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f19c93-80f8-42a8-99c1-f8960ef46f00 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:27.037423 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e50203-b2ed-433f-851d-1084b8d9b2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:27.038019 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e50203-b2ed-433f-851d-1084b8d9b2c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:27.038295 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e50203-b2ed-433f-851d-1084b8d9b2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:27.038563 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e50203-b2ed-433f-851d-1084b8d9b2c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:27.039278 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e50203-b2ed-433f-851d-1084b8d9b2c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:27:27.039576 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e50203-b2ed-433f-851d-1084b8d9b2c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:27.041485 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e50203-b2ed-433f-851d-1084b8d9b2c1 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:27.052883 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1115/2267] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:27:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:27:27.057543 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6583c78d-476a-45f8-b42d-8b3c88fba8df None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:27.066980 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6583c78d-476a-45f8-b42d-8b3c88fba8df None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:27.072638 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b630693d-ddaf-4da1-bc4c-9b2bb6a873f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:27.073254 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630693d-ddaf-4da1-bc4c-9b2bb6a873f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:27.073529 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630693d-ddaf-4da1-bc4c-9b2bb6a873f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:27.073796 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630693d-ddaf-4da1-bc4c-9b2bb6a873f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:27.074488 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b630693d-ddaf-4da1-bc4c-9b2bb6a873f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:27.076529 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b630693d-ddaf-4da1-bc4c-9b2bb6a873f8 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:27.188228 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1153/2268] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:27:27 2022] POST /identity/v3/projects => generated 365 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:27:27.197839 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e50203-b2ed-433f-851d-1084b8d9b2c1 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:27.203893 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e50203-b2ed-433f-851d-1084b8d9b2c1 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:27.272157 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de69855-e3de-4117-8259-3bec79ed8a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:27.272802 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de69855-e3de-4117-8259-3bec79ed8a35 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:27.273540 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de69855-e3de-4117-8259-3bec79ed8a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:27.273540 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de69855-e3de-4117-8259-3bec79ed8a35 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:27.274020 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de69855-e3de-4117-8259-3bec79ed8a35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:27.276927 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de69855-e3de-4117-8259-3bec79ed8a35 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:27.352443 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1116/2269] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:27:27 2022] POST /identity/v3/users => generated 283 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:27:27.356206 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b630693d-ddaf-4da1-bc4c-9b2bb6a873f8 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:27.366544 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b630693d-ddaf-4da1-bc4c-9b2bb6a873f8 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:27.371796 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470bcbb1-9427-4e80-a2c4-623831598376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:27.372405 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470bcbb1-9427-4e80-a2c4-623831598376 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:27.372717 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470bcbb1-9427-4e80-a2c4-623831598376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:27.372986 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470bcbb1-9427-4e80-a2c4-623831598376 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:27.373671 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470bcbb1-9427-4e80-a2c4-623831598376 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:27.375239 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470bcbb1-9427-4e80-a2c4-623831598376 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:27.385744 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1154/2270] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:27:27.390011 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de69855-e3de-4117-8259-3bec79ed8a35 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:27.400888 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de69855-e3de-4117-8259-3bec79ed8a35 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:27.405465 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9e97d-639a-4fdb-b523-d6d4522db336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:27.406111 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9e97d-639a-4fdb-b523-d6d4522db336 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:27.406470 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9e97d-639a-4fdb-b523-d6d4522db336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:27.407015 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9e97d-639a-4fdb-b523-d6d4522db336 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:27.407982 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd9e97d-639a-4fdb-b523-d6d4522db336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:27.410426 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd9e97d-639a-4fdb-b523-d6d4522db336 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:27.422226 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1117/2271] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:27:27.426220 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470bcbb1-9427-4e80-a2c4-623831598376 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:27.436035 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470bcbb1-9427-4e80-a2c4-623831598376 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:27.440639 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feace97-613b-4adc-8950-efa911c0179c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:27.441290 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feace97-613b-4adc-8950-efa911c0179c admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:27.441570 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feace97-613b-4adc-8950-efa911c0179c admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:27.441844 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feace97-613b-4adc-8950-efa911c0179c admin admin] PATH_INFO: `/v3/projects/6d19c4cd8cf94027b6d9c1acba0e549a/users/f894c4143c754f1987ba0e82f6bc390b/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:27.456337 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feace97-613b-4adc-8950-efa911c0179c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d19c4cd8cf94027b6d9c1acba0e549a, user_id=f894c4143c754f1987ba0e82f6bc390b, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:27.458999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feace97-613b-4adc-8950-efa911c0179c admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:27.518081 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1eaa37-7ef8-4a04-8823-0e34d6c2abe3 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:27.518547 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1eaa37-7ef8-4a04-8823-0e34d6c2abe3 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:27.518624 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1eaa37-7ef8-4a04-8823-0e34d6c2abe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:27.534040 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1155/2272] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:27:27 2022] PUT /identity/v3/projects/6d19c4cd8cf94027b6d9c1acba0e549a/users/f894c4143c754f1987ba0e82f6bc390b/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:27:27.538539 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-8c1eaa37-7ef8-4a04-8823-0e34d6c2abe3 None None] MFA Rules not processed for user `17e3b509a2de4159bf234594573f5e18`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:27:27.539768 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8c1eaa37-7ef8-4a04-8823-0e34d6c2abe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:27.544993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4fd09-1486-472b-834a-7a48a6be003d None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:27.545276 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4fd09-1486-472b-834a-7a48a6be003d None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:27.545547 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4fd09-1486-472b-834a-7a48a6be003d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:27.573134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-ced4fd09-1486-472b-834a-7a48a6be003d None None] MFA Rules not processed for user `f894c4143c754f1987ba0e82f6bc390b`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:27:27.574360 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-ced4fd09-1486-472b-834a-7a48a6be003d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:27.585872 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:27:27.599969 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1118/2273] 10.222.0.26 () {58 vars in 967 bytes} [Fri Aug 5 14:27:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:27:27.612894 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1eaa37-7ef8-4a04-8823-0e34d6c2abe3 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:27.615229 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8c1eaa37-7ef8-4a04-8823-0e34d6c2abe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:27.637614 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:27:27.641457 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1156/2274] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:27:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:27:27.657516 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4fd09-1486-472b-834a-7a48a6be003d None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:27.658958 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-ced4fd09-1486-472b-834a-7a48a6be003d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:27.696588 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1eaa37-7ef8-4a04-8823-0e34d6c2abe3 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:27.703322 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17471548-3b59-4319-9826-f667a41ef1b5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dddd7eea67f34f0cbb5f6a1cd85f1fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:27.703946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17471548-3b59-4319-9826-f667a41ef1b5 service swift] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:27.704225 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17471548-3b59-4319-9826-f667a41ef1b5 service swift] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:27.704522 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17471548-3b59-4319-9826-f667a41ef1b5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:27.713980 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17471548-3b59-4319-9826-f667a41ef1b5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:27.715599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17471548-3b59-4319-9826-f667a41ef1b5 service swift] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:27.721850 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4fd09-1486-472b-834a-7a48a6be003d None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:27.724375 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1119/2275] 10.222.0.26 () {60 vars in 1384 bytes} [Fri Aug 5 14:27:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 05 14:27:27.725723 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee724151-e27f-4f5f-9462-d04b3aa2c866 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:27.726355 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee724151-e27f-4f5f-9462-d04b3aa2c866 service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:27.726579 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee724151-e27f-4f5f-9462-d04b3aa2c866 service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:27.726860 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee724151-e27f-4f5f-9462-d04b3aa2c866 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:27.736734 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee724151-e27f-4f5f-9462-d04b3aa2c866 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:27.741902 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee724151-e27f-4f5f-9462-d04b3aa2c866 service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:27.751480 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1157/2276] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:27:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:27:28.424767 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78536c9f-a924-49a9-8733-fa17594ad531 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:28.425053 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78536c9f-a924-49a9-8733-fa17594ad531 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:28.430402 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78536c9f-a924-49a9-8733-fa17594ad531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:28.445128 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-78536c9f-a924-49a9-8733-fa17594ad531 None None] MFA Rules not processed for user `17e3b509a2de4159bf234594573f5e18`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:27:28.446381 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-78536c9f-a924-49a9-8733-fa17594ad531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:28.453752 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1120/2277] 10.222.0.26 () {58 vars in 1009 bytes} [Fri Aug 5 14:27:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:27:28.465821 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee724151-e27f-4f5f-9462-d04b3aa2c866 service cinder] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:28.478424 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee724151-e27f-4f5f-9462-d04b3aa2c866 service cinder] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:28.485418 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c355d795-ed24-415a-8523-864749d50570 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dddd7eea67f34f0cbb5f6a1cd85f1fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:28.486233 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355d795-ed24-415a-8523-864749d50570 service swift] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:28.486935 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355d795-ed24-415a-8523-864749d50570 service swift] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:28.487344 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355d795-ed24-415a-8523-864749d50570 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:28.513573 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c355d795-ed24-415a-8523-864749d50570 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:28.516961 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c355d795-ed24-415a-8523-864749d50570 service swift] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:28.540947 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1158/2278] 10.222.0.26 () {60 vars in 1384 bytes} [Fri Aug 5 14:27:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 14:27:29.122110 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b789054-65b4-4040-9a46-57648067a3ab None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:29.122524 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b789054-65b4-4040-9a46-57648067a3ab None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:29.122653 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b789054-65b4-4040-9a46-57648067a3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:29.142819 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-1b789054-65b4-4040-9a46-57648067a3ab None None] MFA Rules not processed for user `17e3b509a2de4159bf234594573f5e18`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:27:29.143933 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-1b789054-65b4-4040-9a46-57648067a3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:29.151864 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1121/2279] 10.222.0.26 () {58 vars in 967 bytes} [Fri Aug 5 14:27:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:27:29.164275 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c355d795-ed24-415a-8523-864749d50570 service swift] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:29.174803 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c355d795-ed24-415a-8523-864749d50570 service swift] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:29.180198 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ec0f5-aac6-4e79-8304-17f3e4eba53b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dddd7eea67f34f0cbb5f6a1cd85f1fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:29.180836 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ec0f5-aac6-4e79-8304-17f3e4eba53b service swift] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:29.181502 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ec0f5-aac6-4e79-8304-17f3e4eba53b service swift] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:29.181928 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ec0f5-aac6-4e79-8304-17f3e4eba53b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:29.190349 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ec0f5-aac6-4e79-8304-17f3e4eba53b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:29.191899 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ec0f5-aac6-4e79-8304-17f3e4eba53b service swift] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:29.200684 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1159/2280] 10.222.0.26 () {60 vars in 1384 bytes} [Fri Aug 5 14:27:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 14:27:38.288724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b789054-65b4-4040-9a46-57648067a3ab None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:38.290147 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-1b789054-65b4-4040-9a46-57648067a3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:38.359295 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b789054-65b4-4040-9a46-57648067a3ab None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:38.363793 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a831f6-9224-4267-a97f-04a0f322173a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:38.364437 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a831f6-9224-4267-a97f-04a0f322173a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:38.364721 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a831f6-9224-4267-a97f-04a0f322173a admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:38.364985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a831f6-9224-4267-a97f-04a0f322173a admin admin] PATH_INFO: `/v3/users/487d7e4081414c2193cb01ff0a202a6b` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:38.366169 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a831f6-9224-4267-a97f-04a0f322173a admin admin] RBAC: Authorizing `identity:delete_user(user_id=487d7e4081414c2193cb01ff0a202a6b)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:38.368060 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a831f6-9224-4267-a97f-04a0f322173a admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:38.542206 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-15a831f6-9224-4267-a97f-04a0f322173a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '487d7e4081414c2193cb01ff0a202a6b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:38.554401 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-15a831f6-9224-4267-a97f-04a0f322173a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '487d7e4081414c2193cb01ff0a202a6b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:38.558153 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:27:38.618336 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-15a831f6-9224-4267-a97f-04a0f322173a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '487d7e4081414c2193cb01ff0a202a6b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:38.618949 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-15a831f6-9224-4267-a97f-04a0f322173a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '487d7e4081414c2193cb01ff0a202a6b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:38.626277 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-15a831f6-9224-4267-a97f-04a0f322173a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '487d7e4081414c2193cb01ff0a202a6b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:38.633427 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1122/2281] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:27:38 2022] DELETE /identity/v3/users/487d7e4081414c2193cb01ff0a202a6b => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:27:38.637972 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affbab47-8eaf-4699-8264-04695787d0f5 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:38.638536 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affbab47-8eaf-4699-8264-04695787d0f5 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:38.638536 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affbab47-8eaf-4699-8264-04695787d0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:38.657802 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-affbab47-8eaf-4699-8264-04695787d0f5 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:27:38.658999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-affbab47-8eaf-4699-8264-04695787d0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:38.722414 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1160/2282] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:27:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:27:38.733164 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a831f6-9224-4267-a97f-04a0f322173a admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:38.734587 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-15a831f6-9224-4267-a97f-04a0f322173a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:38.793376 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a831f6-9224-4267-a97f-04a0f322173a admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:38.797189 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a33d0d-9978-4652-a943-816eb802bb6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:38.797806 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a33d0d-9978-4652-a943-816eb802bb6a service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:38.798080 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a33d0d-9978-4652-a943-816eb802bb6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:38.798361 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a33d0d-9978-4652-a943-816eb802bb6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:38.808337 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a33d0d-9978-4652-a943-816eb802bb6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:38.809794 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a33d0d-9978-4652-a943-816eb802bb6a service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:38.819174 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1123/2283] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:27:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:27:39.066747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affbab47-8eaf-4699-8264-04695787d0f5 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:39.066747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-affbab47-8eaf-4699-8264-04695787d0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:39.077196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affbab47-8eaf-4699-8264-04695787d0f5 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:39.082339 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c67546-6e3c-4c24-9bcb-525000d745ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:39.082737 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c67546-6e3c-4c24-9bcb-525000d745ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:39.083009 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c67546-6e3c-4c24-9bcb-525000d745ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:39.083327 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c67546-6e3c-4c24-9bcb-525000d745ae admin admin] PATH_INFO: `/v3/projects/33905000a2f64751b88bbff67918d252` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:39.084485 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c67546-6e3c-4c24-9bcb-525000d745ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=33905000a2f64751b88bbff67918d252)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:39.086442 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c67546-6e3c-4c24-9bcb-525000d745ae admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:39.192285 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-46c67546-6e3c-4c24-9bcb-525000d745ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33905000a2f64751b88bbff67918d252', 'request_id': 'req-46c67546-6e3c-4c24-9bcb-525000d745ae'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:39.199208 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-46c67546-6e3c-4c24-9bcb-525000d745ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33905000a2f64751b88bbff67918d252', 'request_id': 'req-46c67546-6e3c-4c24-9bcb-525000d745ae'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:39.205177 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-46c67546-6e3c-4c24-9bcb-525000d745ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33905000a2f64751b88bbff67918d252', 'request_id': 'req-46c67546-6e3c-4c24-9bcb-525000d745ae'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:39.315619 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:27:39.320137 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-46c67546-6e3c-4c24-9bcb-525000d745ae admin admin] The token cache is being invalidate because project 33905000a2f64751b88bbff67918d252 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:27:39.320441 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-46c67546-6e3c-4c24-9bcb-525000d745ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:39.321155 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-46c67546-6e3c-4c24-9bcb-525000d745ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:39.322672 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1161/2284] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:27:39 2022] DELETE /identity/v3/projects/33905000a2f64751b88bbff67918d252 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:27:47.621147 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d77169-588d-46e3-860c-5b9e28b2cdf6 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:47.621680 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d77169-588d-46e3-860c-5b9e28b2cdf6 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:47.621680 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d77169-588d-46e3-860c-5b9e28b2cdf6 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:47.623038 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1124/2285] 10.222.0.26 () {54 vars in 921 bytes} [Fri Aug 5 14:27:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:27:47.653622 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d207db-64c0-4dae-a064-0de9517530dc None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:47.654152 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d207db-64c0-4dae-a064-0de9517530dc None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:47.654152 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d207db-64c0-4dae-a064-0de9517530dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:47.679761 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-22d207db-64c0-4dae-a064-0de9517530dc None None] MFA Rules not processed for user `fff761f6f81a46098f94bd2ec2be9c1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:27:47.681039 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-22d207db-64c0-4dae-a064-0de9517530dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:47.746651 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1162/2286] 10.222.0.26 () {58 vars in 1018 bytes} [Fri Aug 5 14:27:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:27:47.757623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d77169-588d-46e3-860c-5b9e28b2cdf6 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:47.759298 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-48d77169-588d-46e3-860c-5b9e28b2cdf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:47.777173 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d77169-588d-46e3-860c-5b9e28b2cdf6 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:47.782359 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42e68f-06ea-45aa-bf7c-a1aff35bc5a4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff761f6f81a46098f94bd2ec2be9c1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:47.782973 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42e68f-06ea-45aa-bf7c-a1aff35bc5a4 service placement] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:47.783248 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42e68f-06ea-45aa-bf7c-a1aff35bc5a4 service placement] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:47.783519 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42e68f-06ea-45aa-bf7c-a1aff35bc5a4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:47.793844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe42e68f-06ea-45aa-bf7c-a1aff35bc5a4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:47.795183 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe42e68f-06ea-45aa-bf7c-a1aff35bc5a4 service placement] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:47.806204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1125/2287] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:27:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:27:48.728689 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d207db-64c0-4dae-a064-0de9517530dc None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:48.730130 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-22d207db-64c0-4dae-a064-0de9517530dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:48.832604 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d207db-64c0-4dae-a064-0de9517530dc None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:48.838703 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f88c7-f73b-4ad5-8938-30167eb935b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:48.839350 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f88c7-f73b-4ad5-8938-30167eb935b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:48.839632 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f88c7-f73b-4ad5-8938-30167eb935b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:48.839910 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f88c7-f73b-4ad5-8938-30167eb935b6 admin admin] PATH_INFO: `/v3/users/f66e78381e9a4af39527e0cacc1043a5` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:48.841175 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1f88c7-f73b-4ad5-8938-30167eb935b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f66e78381e9a4af39527e0cacc1043a5)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:48.843095 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1f88c7-f73b-4ad5-8938-30167eb935b6 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:49.332791 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-ee1f88c7-f73b-4ad5-8938-30167eb935b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f66e78381e9a4af39527e0cacc1043a5'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:49.341424 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-ee1f88c7-f73b-4ad5-8938-30167eb935b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f66e78381e9a4af39527e0cacc1043a5'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:49.342177 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-ee1f88c7-f73b-4ad5-8938-30167eb935b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f66e78381e9a4af39527e0cacc1043a5'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:49.342811 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-ee1f88c7-f73b-4ad5-8938-30167eb935b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f66e78381e9a4af39527e0cacc1043a5'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:49.355347 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-ee1f88c7-f73b-4ad5-8938-30167eb935b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f66e78381e9a4af39527e0cacc1043a5'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:49.535158 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1163/2288] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:27:48 2022] DELETE /identity/v3/users/f66e78381e9a4af39527e0cacc1043a5 => generated 0 bytes in 807 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:27:49.829111 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42e68f-06ea-45aa-bf7c-a1aff35bc5a4 service placement] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:49.830591 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-fe42e68f-06ea-45aa-bf7c-a1aff35bc5a4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:49.840926 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f88c7-f73b-4ad5-8938-30167eb935b6 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:49.842969 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-ee1f88c7-f73b-4ad5-8938-30167eb935b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:49.847148 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42e68f-06ea-45aa-bf7c-a1aff35bc5a4 service placement] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:49.885977 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:27:49.925039 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa05421-78fa-4e75-a742-4cb4e773109e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:49.925869 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa05421-78fa-4e75-a742-4cb4e773109e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:49.926247 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa05421-78fa-4e75-a742-4cb4e773109e admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:49.926621 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa05421-78fa-4e75-a742-4cb4e773109e admin admin] PATH_INFO: `/v3/users/ab9d7613f0124496a0fa959347998ced` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:49.927923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa05421-78fa-4e75-a742-4cb4e773109e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab9d7613f0124496a0fa959347998ced)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:49.929978 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa05421-78fa-4e75-a742-4cb4e773109e admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:49.939543 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f88c7-f73b-4ad5-8938-30167eb935b6 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:49.944729 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63df3f-55aa-40d4-b923-4317c1d7a478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:49.945378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63df3f-55aa-40d4-b923-4317c1d7a478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:49.945653 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63df3f-55aa-40d4-b923-4317c1d7a478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:49.945925 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63df3f-55aa-40d4-b923-4317c1d7a478 admin admin] PATH_INFO: `/v3/projects/74e9d319c5fd41c6a7cbdf35701a1951` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:49.947248 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f63df3f-55aa-40d4-b923-4317c1d7a478 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74e9d319c5fd41c6a7cbdf35701a1951)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:49.949393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f63df3f-55aa-40d4-b923-4317c1d7a478 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:50.115377 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-4f63df3f-55aa-40d4-b923-4317c1d7a478 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74e9d319c5fd41c6a7cbdf35701a1951', 'request_id': 'req-4f63df3f-55aa-40d4-b923-4317c1d7a478'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:50.117821 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-4aa05421-78fa-4e75-a742-4cb4e773109e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab9d7613f0124496a0fa959347998ced'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:50.124521 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-4f63df3f-55aa-40d4-b923-4317c1d7a478 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74e9d319c5fd41c6a7cbdf35701a1951', 'request_id': 'req-4f63df3f-55aa-40d4-b923-4317c1d7a478'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:50.130949 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-4aa05421-78fa-4e75-a742-4cb4e773109e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab9d7613f0124496a0fa959347998ced'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:50.134413 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-4f63df3f-55aa-40d4-b923-4317c1d7a478 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74e9d319c5fd41c6a7cbdf35701a1951', 'request_id': 'req-4f63df3f-55aa-40d4-b923-4317c1d7a478'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:50.207120 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-4aa05421-78fa-4e75-a742-4cb4e773109e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab9d7613f0124496a0fa959347998ced'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:50.207753 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-4aa05421-78fa-4e75-a742-4cb4e773109e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab9d7613f0124496a0fa959347998ced'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:50.215562 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:27:50.218853 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-4aa05421-78fa-4e75-a742-4cb4e773109e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab9d7613f0124496a0fa959347998ced'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:50.225324 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1126/2289] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:27:49 2022] DELETE /identity/v3/users/ab9d7613f0124496a0fa959347998ced => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:27:50.230566 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6a747-414b-41cb-a875-92ddefc769bc None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:50.231086 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6a747-414b-41cb-a875-92ddefc769bc None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:50.231984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6a747-414b-41cb-a875-92ddefc769bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:50.255451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-a3c6a747-414b-41cb-a875-92ddefc769bc None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:27:50.256745 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a3c6a747-414b-41cb-a875-92ddefc769bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:50.265538 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-4f63df3f-55aa-40d4-b923-4317c1d7a478 admin admin] The token cache is being invalidate because project 74e9d319c5fd41c6a7cbdf35701a1951 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:27:50.267526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-4f63df3f-55aa-40d4-b923-4317c1d7a478 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:50.268131 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-4f63df3f-55aa-40d4-b923-4317c1d7a478 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:50.269999 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1164/2290] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:27:49 2022] DELETE /identity/v3/projects/74e9d319c5fd41c6a7cbdf35701a1951 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:27:50.273976 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63df3f-55aa-40d4-b923-4317c1d7a478 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:50.276232 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-4f63df3f-55aa-40d4-b923-4317c1d7a478 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:50.347950 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63df3f-55aa-40d4-b923-4317c1d7a478 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:50.348156 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1127/2291] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:27:50.356439 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc667b-c363-4f6a-8a71-a1473c300885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:50.357193 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc667b-c363-4f6a-8a71-a1473c300885 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:50.357741 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc667b-c363-4f6a-8a71-a1473c300885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:50.357955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc667b-c363-4f6a-8a71-a1473c300885 admin admin] PATH_INFO: `/v3/users/332cf83af4f14e52a9f40de2c187fd3a` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:50.358702 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6a747-414b-41cb-a875-92ddefc769bc None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:50.359191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc667b-c363-4f6a-8a71-a1473c300885 admin admin] RBAC: Authorizing `identity:delete_user(user_id=332cf83af4f14e52a9f40de2c187fd3a)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:50.360127 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a3c6a747-414b-41cb-a875-92ddefc769bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:50.361275 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc667b-c363-4f6a-8a71-a1473c300885 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:50.430189 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6a747-414b-41cb-a875-92ddefc769bc None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:50.434573 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e834d28-4f1a-4322-9265-294f9126cc96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:50.435190 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e834d28-4f1a-4322-9265-294f9126cc96 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:50.435466 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e834d28-4f1a-4322-9265-294f9126cc96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:50.435734 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e834d28-4f1a-4322-9265-294f9126cc96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:50.445924 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e834d28-4f1a-4322-9265-294f9126cc96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:50.447925 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e834d28-4f1a-4322-9265-294f9126cc96 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:50.458497 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1128/2292] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:27:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 14:27:50.592236 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-63dc667b-c363-4f6a-8a71-a1473c300885 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '332cf83af4f14e52a9f40de2c187fd3a'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:50.593582 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:27:50.600500 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-63dc667b-c363-4f6a-8a71-a1473c300885 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '332cf83af4f14e52a9f40de2c187fd3a'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:50.601238 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-63dc667b-c363-4f6a-8a71-a1473c300885 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '332cf83af4f14e52a9f40de2c187fd3a'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:50.601922 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-63dc667b-c363-4f6a-8a71-a1473c300885 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '332cf83af4f14e52a9f40de2c187fd3a'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:50.642625 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-63dc667b-c363-4f6a-8a71-a1473c300885 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '332cf83af4f14e52a9f40de2c187fd3a'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:50.771567 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1165/2293] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:27:50 2022] DELETE /identity/v3/users/332cf83af4f14e52a9f40de2c187fd3a => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:27:50.980567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e834d28-4f1a-4322-9265-294f9126cc96 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:50.982223 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-1e834d28-4f1a-4322-9265-294f9126cc96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:50.995755 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e834d28-4f1a-4322-9265-294f9126cc96 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:51.058686 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc667b-c363-4f6a-8a71-a1473c300885 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:51.060905 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-63dc667b-c363-4f6a-8a71-a1473c300885 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:51.070253 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94cb2e-b5ac-48fc-ab54-49df4704f772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:51.071108 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94cb2e-b5ac-48fc-ab54-49df4704f772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:51.071454 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94cb2e-b5ac-48fc-ab54-49df4704f772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:51.071793 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94cb2e-b5ac-48fc-ab54-49df4704f772 admin admin] PATH_INFO: `/v3/projects/adcf03f157f24ddcb558a243b6174c54` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:51.073827 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed94cb2e-b5ac-48fc-ab54-49df4704f772 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adcf03f157f24ddcb558a243b6174c54)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:51.076315 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed94cb2e-b5ac-48fc-ab54-49df4704f772 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:51.148491 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc667b-c363-4f6a-8a71-a1473c300885 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:51.154352 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce0176a-bf82-48f2-a0f3-8d7ab3637c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:51.154940 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0176a-bf82-48f2-a0f3-8d7ab3637c34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:51.155222 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0176a-bf82-48f2-a0f3-8d7ab3637c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:51.155498 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0176a-bf82-48f2-a0f3-8d7ab3637c34 admin admin] PATH_INFO: `/v3/projects/fd416c88242f4befbdcab6b0b9b4a9b8` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:51.156778 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce0176a-bf82-48f2-a0f3-8d7ab3637c34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd416c88242f4befbdcab6b0b9b4a9b8)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:51.158964 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce0176a-bf82-48f2-a0f3-8d7ab3637c34 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:51.219171 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:27:51.225766 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ed94cb2e-b5ac-48fc-ab54-49df4704f772 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adcf03f157f24ddcb558a243b6174c54', 'request_id': 'req-ed94cb2e-b5ac-48fc-ab54-49df4704f772'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:51.233595 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ed94cb2e-b5ac-48fc-ab54-49df4704f772 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adcf03f157f24ddcb558a243b6174c54', 'request_id': 'req-ed94cb2e-b5ac-48fc-ab54-49df4704f772'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:51.265669 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-1ce0176a-bf82-48f2-a0f3-8d7ab3637c34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd416c88242f4befbdcab6b0b9b4a9b8', 'request_id': 'req-1ce0176a-bf82-48f2-a0f3-8d7ab3637c34'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:51.271852 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-1ce0176a-bf82-48f2-a0f3-8d7ab3637c34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd416c88242f4befbdcab6b0b9b4a9b8', 'request_id': 'req-1ce0176a-bf82-48f2-a0f3-8d7ab3637c34'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:51.278194 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-1ce0176a-bf82-48f2-a0f3-8d7ab3637c34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd416c88242f4befbdcab6b0b9b4a9b8', 'request_id': 'req-1ce0176a-bf82-48f2-a0f3-8d7ab3637c34'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:51.313634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ed94cb2e-b5ac-48fc-ab54-49df4704f772 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adcf03f157f24ddcb558a243b6174c54', 'request_id': 'req-ed94cb2e-b5ac-48fc-ab54-49df4704f772'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:51.324745 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ed94cb2e-b5ac-48fc-ab54-49df4704f772 admin admin] The token cache is being invalidate because project adcf03f157f24ddcb558a243b6174c54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:27:51.325038 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ed94cb2e-b5ac-48fc-ab54-49df4704f772 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:51.325682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ed94cb2e-b5ac-48fc-ab54-49df4704f772 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:51.327261 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1129/2294] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:27:50 2022] DELETE /identity/v3/projects/adcf03f157f24ddcb558a243b6174c54 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:27:51.332698 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05fef7-b07a-4a2c-8363-b7a0307f78e4 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:51.332980 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05fef7-b07a-4a2c-8363-b7a0307f78e4 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:51.333253 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05fef7-b07a-4a2c-8363-b7a0307f78e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:51.355899 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-cc05fef7-b07a-4a2c-8363-b7a0307f78e4 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:27:51.357174 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-cc05fef7-b07a-4a2c-8363-b7a0307f78e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:51.378090 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-1ce0176a-bf82-48f2-a0f3-8d7ab3637c34 admin admin] The token cache is being invalidate because project fd416c88242f4befbdcab6b0b9b4a9b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:27:51.384192 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-1ce0176a-bf82-48f2-a0f3-8d7ab3637c34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:51.384192 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-1ce0176a-bf82-48f2-a0f3-8d7ab3637c34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:51.384192 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1166/2295] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:27:51 2022] DELETE /identity/v3/projects/fd416c88242f4befbdcab6b0b9b4a9b8 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:27:51.386581 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbce28-43fe-467b-a436-9b5e2471a2e5 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:51.387022 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbce28-43fe-467b-a436-9b5e2471a2e5 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:51.387141 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbce28-43fe-467b-a436-9b5e2471a2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:51.409987 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-82bbce28-43fe-467b-a436-9b5e2471a2e5 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:27:51.409987 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-82bbce28-43fe-467b-a436-9b5e2471a2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:51.417577 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1130/2296] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:27:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:27:51.418929 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1167/2297] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:27:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:27:51.428432 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2e6ed-89ad-44ac-bd2c-44168a646d6e None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:51.428719 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2e6ed-89ad-44ac-bd2c-44168a646d6e None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:51.429020 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2e6ed-89ad-44ac-bd2c-44168a646d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:51.451454 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-67d2e6ed-89ad-44ac-bd2c-44168a646d6e None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:27:51.452719 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-67d2e6ed-89ad-44ac-bd2c-44168a646d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:51.461193 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1131/2298] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:27:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:27:51.466428 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bbce28-43fe-467b-a436-9b5e2471a2e5 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:51.481663 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bbce28-43fe-467b-a436-9b5e2471a2e5 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:51.487284 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a7f05-eaaf-4260-9644-585f979550d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:51.487920 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1bdf5-f85d-4299-a263-f200723cb4e8 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:51.488088 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a7f05-eaaf-4260-9644-585f979550d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:51.488250 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1bdf5-f85d-4299-a263-f200723cb4e8 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:51.488354 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a7f05-eaaf-4260-9644-585f979550d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:51.488598 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1bdf5-f85d-4299-a263-f200723cb4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:51.488704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a7f05-eaaf-4260-9644-585f979550d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:51.489369 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53a7f05-eaaf-4260-9644-585f979550d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:27:51.489694 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53a7f05-eaaf-4260-9644-585f979550d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:51.491293 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53a7f05-eaaf-4260-9644-585f979550d2 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:51.506267 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1168/2299] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:27:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:27:51.510843 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a7f05-eaaf-4260-9644-585f979550d2 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:51.513414 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-26d1bdf5-f85d-4299-a263-f200723cb4e8 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:27:51.514941 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-26d1bdf5-f85d-4299-a263-f200723cb4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:51.523421 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1132/2300] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:27:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:27:51.526675 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a7f05-eaaf-4260-9644-585f979550d2 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:51.527505 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1bdf5-f85d-4299-a263-f200723cb4e8 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:51.532867 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82b5eb-6f67-4452-88e5-5ad7300dc769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:51.533054 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82b5eb-6f67-4452-88e5-5ad7300dc769 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:51.533474 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82b5eb-6f67-4452-88e5-5ad7300dc769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:51.533822 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82b5eb-6f67-4452-88e5-5ad7300dc769 admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:51.534699 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f82b5eb-6f67-4452-88e5-5ad7300dc769 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:51.536901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f82b5eb-6f67-4452-88e5-5ad7300dc769 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:51.537220 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1bdf5-f85d-4299-a263-f200723cb4e8 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:51.544752 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e49cc-fb43-4d98-8b76-785d7ffb30b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:51.545451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e49cc-fb43-4d98-8b76-785d7ffb30b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:51.545798 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e49cc-fb43-4d98-8b76-785d7ffb30b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:51.546148 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e49cc-fb43-4d98-8b76-785d7ffb30b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:51.546851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e49cc-fb43-4d98-8b76-785d7ffb30b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:27:51.547153 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e49cc-fb43-4d98-8b76-785d7ffb30b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:51.548771 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e49cc-fb43-4d98-8b76-785d7ffb30b5 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:51.561378 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1133/2301] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:27:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:27:51.570162 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e49cc-fb43-4d98-8b76-785d7ffb30b5 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:51.580359 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e49cc-fb43-4d98-8b76-785d7ffb30b5 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:51.585651 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e003c-ead5-4404-833d-9981b1b89865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:51.589223 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e003c-ead5-4404-833d-9981b1b89865 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:51.589223 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e003c-ead5-4404-833d-9981b1b89865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:51.589223 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e003c-ead5-4404-833d-9981b1b89865 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:51.589223 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01e003c-ead5-4404-833d-9981b1b89865 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:51.589769 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01e003c-ead5-4404-833d-9981b1b89865 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:51.619946 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1169/2302] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:27:51 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:27:51.624091 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82b5eb-6f67-4452-88e5-5ad7300dc769 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:51.636908 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:27:51.686217 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82b5eb-6f67-4452-88e5-5ad7300dc769 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:51.691213 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935692f5-20f9-40bd-8e39-2bd21db162d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:51.691830 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935692f5-20f9-40bd-8e39-2bd21db162d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:51.692108 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935692f5-20f9-40bd-8e39-2bd21db162d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:51.692383 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935692f5-20f9-40bd-8e39-2bd21db162d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:51.693084 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935692f5-20f9-40bd-8e39-2bd21db162d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:51.695182 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935692f5-20f9-40bd-8e39-2bd21db162d0 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:51.704839 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1134/2303] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:27:51 2022] POST /identity/v3/projects => generated 371 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:27:51.709720 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e003c-ead5-4404-833d-9981b1b89865 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:51.772235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e003c-ead5-4404-833d-9981b1b89865 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:51.778072 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb21d90-f758-4b5b-8c5b-85364c3835b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:51.778677 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb21d90-f758-4b5b-8c5b-85364c3835b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:51.778948 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb21d90-f758-4b5b-8c5b-85364c3835b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:51.779214 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb21d90-f758-4b5b-8c5b-85364c3835b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:51.779984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb21d90-f758-4b5b-8c5b-85364c3835b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:51.781902 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb21d90-f758-4b5b-8c5b-85364c3835b2 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:51.793882 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1170/2304] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:27:51 2022] POST /identity/v3/users => generated 287 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:27:51.797964 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935692f5-20f9-40bd-8e39-2bd21db162d0 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:51.808574 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935692f5-20f9-40bd-8e39-2bd21db162d0 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:51.813278 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f3cc8-9385-4636-b5b0-7bd011243de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:51.813889 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f3cc8-9385-4636-b5b0-7bd011243de7 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:51.814166 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f3cc8-9385-4636-b5b0-7bd011243de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:51.814441 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f3cc8-9385-4636-b5b0-7bd011243de7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:51.815142 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383f3cc8-9385-4636-b5b0-7bd011243de7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:51.817369 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383f3cc8-9385-4636-b5b0-7bd011243de7 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:51.828634 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1171/2305] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:27:51.834092 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f3cc8-9385-4636-b5b0-7bd011243de7 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:51.845309 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f3cc8-9385-4636-b5b0-7bd011243de7 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:51.850407 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f73de1-ff5b-47e1-84d0-8555a6dd4be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:51.851152 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f73de1-ff5b-47e1-84d0-8555a6dd4be5 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:51.851503 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f73de1-ff5b-47e1-84d0-8555a6dd4be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:51.851778 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f73de1-ff5b-47e1-84d0-8555a6dd4be5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:51.852644 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f73de1-ff5b-47e1-84d0-8555a6dd4be5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:51.854241 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f73de1-ff5b-47e1-84d0-8555a6dd4be5 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:51.864922 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1172/2306] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:27:51.869734 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f73de1-ff5b-47e1-84d0-8555a6dd4be5 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:51.879524 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f73de1-ff5b-47e1-84d0-8555a6dd4be5 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:51.886437 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47817260-0dc4-4716-844b-55023830975a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:51.887435 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47817260-0dc4-4716-844b-55023830975a admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:51.888850 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47817260-0dc4-4716-844b-55023830975a admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:51.888976 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1135/2307] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:27:51 2022] POST /identity/v3/users => generated 286 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:27:51.889071 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47817260-0dc4-4716-844b-55023830975a admin admin] PATH_INFO: `/v3/projects/f8d2ffcd282c4456891d5a9bbbadd758/users/7cdd3bec678747f78fa35f1e908ec60c/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:51.892946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb21d90-f758-4b5b-8c5b-85364c3835b2 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:51.903491 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb21d90-f758-4b5b-8c5b-85364c3835b2 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:51.905595 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47817260-0dc4-4716-844b-55023830975a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8d2ffcd282c4456891d5a9bbbadd758, user_id=7cdd3bec678747f78fa35f1e908ec60c, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:51.908286 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47817260-0dc4-4716-844b-55023830975a admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:51.909050 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6be539-387e-4213-a8ff-485e031648e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:51.909788 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6be539-387e-4213-a8ff-485e031648e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:51.910131 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6be539-387e-4213-a8ff-485e031648e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:51.910468 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6be539-387e-4213-a8ff-485e031648e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:51.911235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6be539-387e-4213-a8ff-485e031648e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:51.913124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6be539-387e-4213-a8ff-485e031648e1 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:51.918563 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:27:51.923489 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1136/2308] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:27:51.927510 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6be539-387e-4213-a8ff-485e031648e1 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:51.938728 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6be539-387e-4213-a8ff-485e031648e1 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:51.943291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a28a56-0668-4c04-8186-2b7ac60ed8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:51.946440 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a28a56-0668-4c04-8186-2b7ac60ed8f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:51.946440 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a28a56-0668-4c04-8186-2b7ac60ed8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:51.946440 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a28a56-0668-4c04-8186-2b7ac60ed8f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:51.946440 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a28a56-0668-4c04-8186-2b7ac60ed8f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:51.946922 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a28a56-0668-4c04-8186-2b7ac60ed8f8 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:51.957477 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1137/2309] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:27:51.961640 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a28a56-0668-4c04-8186-2b7ac60ed8f8 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:51.971275 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1173/2310] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:27:51 2022] PUT /identity/v3/projects/f8d2ffcd282c4456891d5a9bbbadd758/users/7cdd3bec678747f78fa35f1e908ec60c/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:27:51.972161 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a28a56-0668-4c04-8186-2b7ac60ed8f8 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:51.984980 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc6ea1-bc4a-4433-94d4-6ed9ae533983 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:51.985291 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc6ea1-bc4a-4433-94d4-6ed9ae533983 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:51.985565 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc6ea1-bc4a-4433-94d4-6ed9ae533983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:52.015156 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-7ecc6ea1-bc4a-4433-94d4-6ed9ae533983 None None] MFA Rules not processed for user `7cdd3bec678747f78fa35f1e908ec60c`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:27:52.016589 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-7ecc6ea1-bc4a-4433-94d4-6ed9ae533983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:52.031873 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d979b7d-e072-4d91-8708-a20195ed0624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:52.032626 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d979b7d-e072-4d91-8708-a20195ed0624 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:52.033711 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d979b7d-e072-4d91-8708-a20195ed0624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:52.033711 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d979b7d-e072-4d91-8708-a20195ed0624 admin admin] PATH_INFO: `/v3/projects/4717d08ca01041088bd4b95f00d7b5ef/users/33b2cb4c04f6495bbb5ff171df953ef0/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:52.048455 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d979b7d-e072-4d91-8708-a20195ed0624 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4717d08ca01041088bd4b95f00d7b5ef, user_id=33b2cb4c04f6495bbb5ff171df953ef0, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:52.051137 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d979b7d-e072-4d91-8708-a20195ed0624 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:52.082572 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1174/2311] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:27:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:27:52.094604 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc6ea1-bc4a-4433-94d4-6ed9ae533983 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:52.096022 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-7ecc6ea1-bc4a-4433-94d4-6ed9ae533983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:52.099243 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1138/2312] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:27:51 2022] PUT /identity/v3/projects/4717d08ca01041088bd4b95f00d7b5ef/users/33b2cb4c04f6495bbb5ff171df953ef0/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:27:52.104776 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f275548-f40f-4b15-9b8b-a7462af8467b None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:52.105141 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f275548-f40f-4b15-9b8b-a7462af8467b None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:52.105417 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f275548-f40f-4b15-9b8b-a7462af8467b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:52.123766 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-4f275548-f40f-4b15-9b8b-a7462af8467b None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:27:52.125090 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-4f275548-f40f-4b15-9b8b-a7462af8467b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:52.147361 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:27:52.153427 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc6ea1-bc4a-4433-94d4-6ed9ae533983 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:52.184284 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1139/2313] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:27:52.194306 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f275548-f40f-4b15-9b8b-a7462af8467b None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:52.195202 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-4f275548-f40f-4b15-9b8b-a7462af8467b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:52.202862 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd405f50-93c3-40eb-82da-646323c92677 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:52.203557 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd405f50-93c3-40eb-82da-646323c92677 service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:52.203841 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd405f50-93c3-40eb-82da-646323c92677 service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:52.204113 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd405f50-93c3-40eb-82da-646323c92677 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:52.252375 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f275548-f40f-4b15-9b8b-a7462af8467b None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:52.256100 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccde86f-59bd-40ff-8ed4-adfc0bf45d0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:52.256754 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccde86f-59bd-40ff-8ed4-adfc0bf45d0d service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:52.257026 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccde86f-59bd-40ff-8ed4-adfc0bf45d0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:52.257294 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccde86f-59bd-40ff-8ed4-adfc0bf45d0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:52.262401 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd405f50-93c3-40eb-82da-646323c92677 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:52.263855 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd405f50-93c3-40eb-82da-646323c92677 service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:52.267575 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccde86f-59bd-40ff-8ed4-adfc0bf45d0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:52.269027 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccde86f-59bd-40ff-8ed4-adfc0bf45d0d service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:52.272722 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1175/2314] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:27:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 14:27:52.279563 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1140/2315] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:27:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 14:27:52.298969 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd405f50-93c3-40eb-82da-646323c92677 service cinder] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:52.310290 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd405f50-93c3-40eb-82da-646323c92677 service cinder] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:52.315001 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5f1ff-3173-44a0-b3ba-9b568aca773e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:52.315639 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5f1ff-3173-44a0-b3ba-9b568aca773e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:52.315919 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5f1ff-3173-44a0-b3ba-9b568aca773e admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:52.316191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5f1ff-3173-44a0-b3ba-9b568aca773e admin admin] PATH_INFO: `/v3/users/7cdd3bec678747f78fa35f1e908ec60c` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:52.317389 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d5f1ff-3173-44a0-b3ba-9b568aca773e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cdd3bec678747f78fa35f1e908ec60c)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:52.319290 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d5f1ff-3173-44a0-b3ba-9b568aca773e admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:52.528592 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-78d5f1ff-3173-44a0-b3ba-9b568aca773e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cdd3bec678747f78fa35f1e908ec60c'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:52.536549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-78d5f1ff-3173-44a0-b3ba-9b568aca773e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cdd3bec678747f78fa35f1e908ec60c'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:52.539187 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-78d5f1ff-3173-44a0-b3ba-9b568aca773e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cdd3bec678747f78fa35f1e908ec60c'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:52.539187 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-78d5f1ff-3173-44a0-b3ba-9b568aca773e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cdd3bec678747f78fa35f1e908ec60c'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:52.552171 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-78d5f1ff-3173-44a0-b3ba-9b568aca773e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cdd3bec678747f78fa35f1e908ec60c'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:52.640841 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1176/2316] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:27:52 2022] DELETE /identity/v3/users/7cdd3bec678747f78fa35f1e908ec60c => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:27:52.645394 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7440f9-2433-45dc-bc42-bcb647d44285 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:52.646149 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7440f9-2433-45dc-bc42-bcb647d44285 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:52.646149 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7440f9-2433-45dc-bc42-bcb647d44285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:52.666636 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-bb7440f9-2433-45dc-bc42-bcb647d44285 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:27:52.667983 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-bb7440f9-2433-45dc-bc42-bcb647d44285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:52.677251 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:27:52.730291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1141/2317] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:27:52.740539 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5f1ff-3173-44a0-b3ba-9b568aca773e admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:52.743025 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-78d5f1ff-3173-44a0-b3ba-9b568aca773e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:52.781899 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:27:52.820458 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5f1ff-3173-44a0-b3ba-9b568aca773e admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:52.826063 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee80d9c-aa35-4787-aefe-4de5d2f619f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:52.827367 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee80d9c-aa35-4787-aefe-4de5d2f619f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:52.827664 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee80d9c-aa35-4787-aefe-4de5d2f619f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:52.827981 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee80d9c-aa35-4787-aefe-4de5d2f619f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:52.840811 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee80d9c-aa35-4787-aefe-4de5d2f619f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:52.842441 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee80d9c-aa35-4787-aefe-4de5d2f619f9 service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:52.852593 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1177/2318] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:27:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:27:53.417298 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7440f9-2433-45dc-bc42-bcb647d44285 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:53.418917 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-bb7440f9-2433-45dc-bc42-bcb647d44285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:53.434828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7440f9-2433-45dc-bc42-bcb647d44285 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:53.441387 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a705c7-ae7e-4f56-9a6e-3582f4c316c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:53.442025 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a705c7-ae7e-4f56-9a6e-3582f4c316c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:53.442863 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a705c7-ae7e-4f56-9a6e-3582f4c316c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:53.443138 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a705c7-ae7e-4f56-9a6e-3582f4c316c8 admin admin] PATH_INFO: `/v3/projects/f8d2ffcd282c4456891d5a9bbbadd758` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:53.444616 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a705c7-ae7e-4f56-9a6e-3582f4c316c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8d2ffcd282c4456891d5a9bbbadd758)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:53.446472 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a705c7-ae7e-4f56-9a6e-3582f4c316c8 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:53.561460 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e5a705c7-ae7e-4f56-9a6e-3582f4c316c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8d2ffcd282c4456891d5a9bbbadd758', 'request_id': 'req-e5a705c7-ae7e-4f56-9a6e-3582f4c316c8'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:53.568591 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e5a705c7-ae7e-4f56-9a6e-3582f4c316c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8d2ffcd282c4456891d5a9bbbadd758', 'request_id': 'req-e5a705c7-ae7e-4f56-9a6e-3582f4c316c8'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:53.646626 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e5a705c7-ae7e-4f56-9a6e-3582f4c316c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8d2ffcd282c4456891d5a9bbbadd758', 'request_id': 'req-e5a705c7-ae7e-4f56-9a6e-3582f4c316c8'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:53.659073 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e5a705c7-ae7e-4f56-9a6e-3582f4c316c8 admin admin] The token cache is being invalidate because project f8d2ffcd282c4456891d5a9bbbadd758 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:27:53.659430 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e5a705c7-ae7e-4f56-9a6e-3582f4c316c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:53.660119 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e5a705c7-ae7e-4f56-9a6e-3582f4c316c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:27:53.661809 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1142/2319] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:27:53 2022] DELETE /identity/v3/projects/f8d2ffcd282c4456891d5a9bbbadd758 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:27:53.667311 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e3573-85e3-4c8d-a004-0df25dd461be None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:53.667796 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e3573-85e3-4c8d-a004-0df25dd461be None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:53.667923 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e3573-85e3-4c8d-a004-0df25dd461be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:53.687693 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-b08e3573-85e3-4c8d-a004-0df25dd461be None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:27:53.689086 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-b08e3573-85e3-4c8d-a004-0df25dd461be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:53.762066 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1178/2320] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:27:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:27:53.769660 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d4ad1-8826-4132-bca8-72b6e610c3b4 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:53.769938 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d4ad1-8826-4132-bca8-72b6e610c3b4 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:53.770211 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d4ad1-8826-4132-bca8-72b6e610c3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:53.797830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-492d4ad1-8826-4132-bca8-72b6e610c3b4 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:27:53.797830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-492d4ad1-8826-4132-bca8-72b6e610c3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:53.807991 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1143/2321] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:27:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:27:53.812381 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e3573-85e3-4c8d-a004-0df25dd461be None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:53.826865 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e3573-85e3-4c8d-a004-0df25dd461be None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:53.833138 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ee0a1-b56f-4f4c-b379-3768e841d178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:53.833901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ee0a1-b56f-4f4c-b379-3768e841d178 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:53.834232 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ee0a1-b56f-4f4c-b379-3768e841d178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:53.834845 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ee0a1-b56f-4f4c-b379-3768e841d178 admin admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:53.836672 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0ee0a1-b56f-4f4c-b379-3768e841d178 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:27:53.836846 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0ee0a1-b56f-4f4c-b379-3768e841d178 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:53.838846 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0ee0a1-b56f-4f4c-b379-3768e841d178 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:53.855438 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1179/2322] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:27:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:27:53.862135 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492d4ad1-8826-4132-bca8-72b6e610c3b4 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:53.872321 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492d4ad1-8826-4132-bca8-72b6e610c3b4 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:53.877092 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9925aefb-19cb-44cf-98ea-e1a864769e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:53.877767 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925aefb-19cb-44cf-98ea-e1a864769e65 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:53.877983 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925aefb-19cb-44cf-98ea-e1a864769e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:53.878252 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925aefb-19cb-44cf-98ea-e1a864769e65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:53.878915 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9925aefb-19cb-44cf-98ea-e1a864769e65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:53.880848 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9925aefb-19cb-44cf-98ea-e1a864769e65 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:53.976450 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1144/2323] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:27:53 2022] POST /identity/v3/projects => generated 375 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:27:53.980631 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ee0a1-b56f-4f4c-b379-3768e841d178 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:54.002786 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ee0a1-b56f-4f4c-b379-3768e841d178 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:54.078704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a8aae-2103-4033-b615-bc504c941476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:54.079503 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a8aae-2103-4033-b615-bc504c941476 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:54.079792 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a8aae-2103-4033-b615-bc504c941476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:54.080068 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a8aae-2103-4033-b615-bc504c941476 admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:54.080784 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6a8aae-2103-4033-b615-bc504c941476 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:54.082815 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6a8aae-2103-4033-b615-bc504c941476 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:54.194727 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1180/2324] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:27:53 2022] POST /identity/v3/users => generated 288 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:27:54.195383 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9925aefb-19cb-44cf-98ea-e1a864769e65 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:54.208770 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9925aefb-19cb-44cf-98ea-e1a864769e65 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:54.214033 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a0199-3c0a-4003-820f-1a7d03da2f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:54.214647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a0199-3c0a-4003-820f-1a7d03da2f1c admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:54.214928 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a0199-3c0a-4003-820f-1a7d03da2f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:54.215531 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a0199-3c0a-4003-820f-1a7d03da2f1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:54.215896 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62a0199-3c0a-4003-820f-1a7d03da2f1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:54.217544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62a0199-3c0a-4003-820f-1a7d03da2f1c admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:54.229026 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1145/2325] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:27:54.233372 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a8aae-2103-4033-b615-bc504c941476 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:54.245774 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a8aae-2103-4033-b615-bc504c941476 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:54.251113 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e8ae48-2023-4c60-ac0f-eed70278c515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:54.251579 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8ae48-2023-4c60-ac0f-eed70278c515 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:54.251950 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8ae48-2023-4c60-ac0f-eed70278c515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:54.252320 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8ae48-2023-4c60-ac0f-eed70278c515 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:54.253164 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e8ae48-2023-4c60-ac0f-eed70278c515 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:54.254904 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e8ae48-2023-4c60-ac0f-eed70278c515 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:54.267540 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1181/2326] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:27:54.271776 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a0199-3c0a-4003-820f-1a7d03da2f1c admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:54.284894 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a0199-3c0a-4003-820f-1a7d03da2f1c admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:54.289496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f09727-6cf8-4a63-87b1-6e0133fda2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:54.290145 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f09727-6cf8-4a63-87b1-6e0133fda2b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:54.290421 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f09727-6cf8-4a63-87b1-6e0133fda2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:54.291714 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f09727-6cf8-4a63-87b1-6e0133fda2b6 admin admin] PATH_INFO: `/v3/projects/37e4e6d405de48378da3fcc67fca8fb2/users/9c43121a82de490ebbffa006dcc11f07/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:54.306870 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f09727-6cf8-4a63-87b1-6e0133fda2b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37e4e6d405de48378da3fcc67fca8fb2, user_id=9c43121a82de490ebbffa006dcc11f07, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:54.310757 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f09727-6cf8-4a63-87b1-6e0133fda2b6 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:54.434993 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1146/2327] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:27:54 2022] PUT /identity/v3/projects/37e4e6d405de48378da3fcc67fca8fb2/users/9c43121a82de490ebbffa006dcc11f07/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:27:54.446854 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78bc0e-5c03-4db3-9144-c2a704593a3c None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:54.447368 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78bc0e-5c03-4db3-9144-c2a704593a3c None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:54.447949 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78bc0e-5c03-4db3-9144-c2a704593a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:54.517563 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-3d78bc0e-5c03-4db3-9144-c2a704593a3c None None] MFA Rules not processed for user `9c43121a82de490ebbffa006dcc11f07`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:27:54.518879 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-3d78bc0e-5c03-4db3-9144-c2a704593a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:54.555923 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:27:54.611772 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1182/2328] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:27:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:27:54.618737 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f09727-6cf8-4a63-87b1-6e0133fda2b6 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:54.621480 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-52f09727-6cf8-4a63-87b1-6e0133fda2b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:54.664988 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:27:54.681713 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f09727-6cf8-4a63-87b1-6e0133fda2b6 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:54.685684 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b25f5a-1297-475b-82cc-621736feda45 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1ce2a2b4eb405b999d3fb756795176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:54.686323 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b25f5a-1297-475b-82cc-621736feda45 service glance] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:54.686601 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b25f5a-1297-475b-82cc-621736feda45 service glance] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:54.686865 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b25f5a-1297-475b-82cc-621736feda45 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:54.695402 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b25f5a-1297-475b-82cc-621736feda45 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:54.697888 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b25f5a-1297-475b-82cc-621736feda45 service glance] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:54.706742 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1147/2329] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:27:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:27:58.030326 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f4f59-1236-4cf9-8efa-1400f2df21c2 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:58.030820 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f4f59-1236-4cf9-8efa-1400f2df21c2 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:58.030901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f4f59-1236-4cf9-8efa-1400f2df21c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:58.062395 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-882f4f59-1236-4cf9-8efa-1400f2df21c2 None None] MFA Rules not processed for user `33b2cb4c04f6495bbb5ff171df953ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:27:58.063526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-882f4f59-1236-4cf9-8efa-1400f2df21c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:58.145068 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1183/2330] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:27:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:27:58.146316 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b25f5a-1297-475b-82cc-621736feda45 service glance] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:58.147952 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-06b25f5a-1297-475b-82cc-621736feda45 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:27:58.223806 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b25f5a-1297-475b-82cc-621736feda45 service glance] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:58.229938 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3078eb-2796-44f9-889c-e7bb46845ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:58.234871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3078eb-2796-44f9-889c-e7bb46845ce4 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:58.234871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3078eb-2796-44f9-889c-e7bb46845ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:58.234871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3078eb-2796-44f9-889c-e7bb46845ce4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:58.234871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3078eb-2796-44f9-889c-e7bb46845ce4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:58.234871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3078eb-2796-44f9-889c-e7bb46845ce4 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:58.366381 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1148/2331] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:27:58 2022] POST /identity/v3/projects => generated 373 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:27:58.371231 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f4f59-1236-4cf9-8efa-1400f2df21c2 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:58.382429 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f4f59-1236-4cf9-8efa-1400f2df21c2 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:58.457652 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f83f2b7-d580-41ef-9ef7-e596ab11797a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:58.458325 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83f2b7-d580-41ef-9ef7-e596ab11797a admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:58.458604 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83f2b7-d580-41ef-9ef7-e596ab11797a admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:58.458876 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83f2b7-d580-41ef-9ef7-e596ab11797a admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:58.459552 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f83f2b7-d580-41ef-9ef7-e596ab11797a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:58.461549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f83f2b7-d580-41ef-9ef7-e596ab11797a admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:58.544353 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1184/2332] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:27:58 2022] POST /identity/v3/users => generated 293 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:27:58.548140 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3078eb-2796-44f9-889c-e7bb46845ce4 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:58.560035 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3078eb-2796-44f9-889c-e7bb46845ce4 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:58.564682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de56cccd-9e33-47a2-ace2-c017f69094ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:58.565299 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56cccd-9e33-47a2-ace2-c017f69094ec admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:58.565579 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56cccd-9e33-47a2-ace2-c017f69094ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:58.565852 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56cccd-9e33-47a2-ace2-c017f69094ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:58.566540 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de56cccd-9e33-47a2-ace2-c017f69094ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:58.568196 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de56cccd-9e33-47a2-ace2-c017f69094ec admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:58.580793 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1149/2333] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:27:58.584964 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f83f2b7-d580-41ef-9ef7-e596ab11797a admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:58.595931 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f83f2b7-d580-41ef-9ef7-e596ab11797a admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:58.600618 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d208d9b-d8b3-4965-90dc-21a7f3288425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:58.601326 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d208d9b-d8b3-4965-90dc-21a7f3288425 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:58.604194 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d208d9b-d8b3-4965-90dc-21a7f3288425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:58.604194 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d208d9b-d8b3-4965-90dc-21a7f3288425 admin admin] PATH_INFO: `/v3/projects/ccaa8dabff724f9ca0b296a92b4bfc70/users/2806cf9ecdcb410aad9d0c4710ae43dc/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:58.617704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d208d9b-d8b3-4965-90dc-21a7f3288425 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccaa8dabff724f9ca0b296a92b4bfc70, user_id=2806cf9ecdcb410aad9d0c4710ae43dc, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:58.621343 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d208d9b-d8b3-4965-90dc-21a7f3288425 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:58.720813 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1185/2334] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:27:58 2022] PUT /identity/v3/projects/ccaa8dabff724f9ca0b296a92b4bfc70/users/2806cf9ecdcb410aad9d0c4710ae43dc/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:27:58.724883 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de56cccd-9e33-47a2-ace2-c017f69094ec admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:58.735686 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de56cccd-9e33-47a2-ace2-c017f69094ec admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:58.799432 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf4decc-a316-4f69-a64e-a14247643198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:58.800149 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4decc-a316-4f69-a64e-a14247643198 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:58.800454 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4decc-a316-4f69-a64e-a14247643198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:58.800733 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4decc-a316-4f69-a64e-a14247643198 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:58.801434 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf4decc-a316-4f69-a64e-a14247643198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:58.803026 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf4decc-a316-4f69-a64e-a14247643198 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:58.813777 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1150/2335] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:27:58.817947 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d208d9b-d8b3-4965-90dc-21a7f3288425 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:27:58.828655 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d208d9b-d8b3-4965-90dc-21a7f3288425 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:27:58.833203 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636795e5-d4a0-4abe-a8e0-3f545f87bcea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:27:58.833858 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636795e5-d4a0-4abe-a8e0-3f545f87bcea admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:27:58.834134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636795e5-d4a0-4abe-a8e0-3f545f87bcea admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:27:58.834406 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636795e5-d4a0-4abe-a8e0-3f545f87bcea admin admin] PATH_INFO: `/v3/projects/ccaa8dabff724f9ca0b296a92b4bfc70/users/2806cf9ecdcb410aad9d0c4710ae43dc/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:27:58.836311 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636795e5-d4a0-4abe-a8e0-3f545f87bcea admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccaa8dabff724f9ca0b296a92b4bfc70, user_id=2806cf9ecdcb410aad9d0c4710ae43dc, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:27:58.840249 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636795e5-d4a0-4abe-a8e0-3f545f87bcea admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:27:58.850259 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1186/2336] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:27:58 2022] PUT /identity/v3/projects/ccaa8dabff724f9ca0b296a92b4bfc70/users/2806cf9ecdcb410aad9d0c4710ae43dc/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:28:04.866128 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f9778-96b6-44e3-b906-5b820cc179b0 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:04.866722 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f9778-96b6-44e3-b906-5b820cc179b0 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:04.866722 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f9778-96b6-44e3-b906-5b820cc179b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:04.895593 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-6b8f9778-96b6-44e3-b906-5b820cc179b0 None None] MFA Rules not processed for user `2806cf9ecdcb410aad9d0c4710ae43dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:28:04.896926 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-6b8f9778-96b6-44e3-b906-5b820cc179b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:04.949282 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:28:04.996997 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1151/2337] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:28:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:28:05.015368 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636795e5-d4a0-4abe-a8e0-3f545f87bcea admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:05.017345 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-636795e5-d4a0-4abe-a8e0-3f545f87bcea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:05.029344 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:28:05.102220 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636795e5-d4a0-4abe-a8e0-3f545f87bcea admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:05.110155 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec72f6a-0ec6-4b04-9e08-b82db0e921f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:05.110816 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec72f6a-0ec6-4b04-9e08-b82db0e921f8 service nova] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:05.111144 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec72f6a-0ec6-4b04-9e08-b82db0e921f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:05.111380 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec72f6a-0ec6-4b04-9e08-b82db0e921f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:05.189234 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec72f6a-0ec6-4b04-9e08-b82db0e921f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:05.191461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec72f6a-0ec6-4b04-9e08-b82db0e921f8 service nova] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:05.212827 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1187/2338] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:28:05 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:28:07.791660 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21548f-835d-4577-a334-bd67062b68a8 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:07.792139 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21548f-835d-4577-a334-bd67062b68a8 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:07.792252 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21548f-835d-4577-a334-bd67062b68a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:07.812372 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-af21548f-835d-4577-a334-bd67062b68a8 None None] MFA Rules not processed for user `33b2cb4c04f6495bbb5ff171df953ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:28:07.813765 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-af21548f-835d-4577-a334-bd67062b68a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:07.820789 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1152/2339] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:28:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:28:15.214761 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=52 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:28:17.807369 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=15 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:28:40.501431 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec72f6a-0ec6-4b04-9e08-b82db0e921f8 service nova] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:40.561574 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec72f6a-0ec6-4b04-9e08-b82db0e921f8 service nova] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:40.566690 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d725c-10b8-4b38-8661-e86464d1aeb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:40.567321 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d725c-10b8-4b38-8661-e86464d1aeb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:40.567595 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d725c-10b8-4b38-8661-e86464d1aeb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:40.567867 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d725c-10b8-4b38-8661-e86464d1aeb1 admin admin] PATH_INFO: `/v3/users/9c43121a82de490ebbffa006dcc11f07` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:40.569059 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d725c-10b8-4b38-8661-e86464d1aeb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c43121a82de490ebbffa006dcc11f07)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:40.570977 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d725c-10b8-4b38-8661-e86464d1aeb1 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:40.765265 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-ab6d725c-10b8-4b38-8661-e86464d1aeb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c43121a82de490ebbffa006dcc11f07'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:40.771939 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-ab6d725c-10b8-4b38-8661-e86464d1aeb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c43121a82de490ebbffa006dcc11f07'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:40.772632 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-ab6d725c-10b8-4b38-8661-e86464d1aeb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c43121a82de490ebbffa006dcc11f07'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:40.773247 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-ab6d725c-10b8-4b38-8661-e86464d1aeb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c43121a82de490ebbffa006dcc11f07'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:40.785024 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-ab6d725c-10b8-4b38-8661-e86464d1aeb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c43121a82de490ebbffa006dcc11f07'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:40.871369 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1188/2340] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:28:40 2022] DELETE /identity/v3/users/9c43121a82de490ebbffa006dcc11f07 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:28:40.876502 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb5881-13f4-4096-a730-aa35358ac5c2 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:40.877040 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb5881-13f4-4096-a730-aa35358ac5c2 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:40.877157 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb5881-13f4-4096-a730-aa35358ac5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:40.896570 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-efbb5881-13f4-4096-a730-aa35358ac5c2 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:28:40.897756 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-efbb5881-13f4-4096-a730-aa35358ac5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:40.955886 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1153/2341] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:28:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:28:40.966112 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d725c-10b8-4b38-8661-e86464d1aeb1 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:40.967525 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-ab6d725c-10b8-4b38-8661-e86464d1aeb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:41.024382 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d725c-10b8-4b38-8661-e86464d1aeb1 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:41.028054 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2b4ab2-43f9-4fe8-b646-8c8852e6ae02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:41.028693 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2b4ab2-43f9-4fe8-b646-8c8852e6ae02 service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:41.028975 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2b4ab2-43f9-4fe8-b646-8c8852e6ae02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:41.029249 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2b4ab2-43f9-4fe8-b646-8c8852e6ae02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:41.038901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2b4ab2-43f9-4fe8-b646-8c8852e6ae02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:41.040319 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2b4ab2-43f9-4fe8-b646-8c8852e6ae02 service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:41.050220 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1189/2342] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:28:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:28:41.533720 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb5881-13f4-4096-a730-aa35358ac5c2 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:41.535546 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-efbb5881-13f4-4096-a730-aa35358ac5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:41.550210 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb5881-13f4-4096-a730-aa35358ac5c2 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:41.555173 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ea830-aec4-475f-b6c1-9dbabb5a59d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:41.556086 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ea830-aec4-475f-b6c1-9dbabb5a59d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:41.556359 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ea830-aec4-475f-b6c1-9dbabb5a59d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:41.556661 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ea830-aec4-475f-b6c1-9dbabb5a59d5 admin admin] PATH_INFO: `/v3/projects/37e4e6d405de48378da3fcc67fca8fb2` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:41.557885 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ea830-aec4-475f-b6c1-9dbabb5a59d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37e4e6d405de48378da3fcc67fca8fb2)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:41.560259 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ea830-aec4-475f-b6c1-9dbabb5a59d5 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:41.692618 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-264ea830-aec4-475f-b6c1-9dbabb5a59d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37e4e6d405de48378da3fcc67fca8fb2', 'request_id': 'req-264ea830-aec4-475f-b6c1-9dbabb5a59d5'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:41.699888 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-264ea830-aec4-475f-b6c1-9dbabb5a59d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37e4e6d405de48378da3fcc67fca8fb2', 'request_id': 'req-264ea830-aec4-475f-b6c1-9dbabb5a59d5'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:41.793490 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-264ea830-aec4-475f-b6c1-9dbabb5a59d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37e4e6d405de48378da3fcc67fca8fb2', 'request_id': 'req-264ea830-aec4-475f-b6c1-9dbabb5a59d5'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:41.804910 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-264ea830-aec4-475f-b6c1-9dbabb5a59d5 admin admin] The token cache is being invalidate because project 37e4e6d405de48378da3fcc67fca8fb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:28:41.805205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-264ea830-aec4-475f-b6c1-9dbabb5a59d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:41.805858 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-264ea830-aec4-475f-b6c1-9dbabb5a59d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:41.807362 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1154/2343] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:28:41 2022] DELETE /identity/v3/projects/37e4e6d405de48378da3fcc67fca8fb2 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:28:41.812307 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cab82b-da35-4d8c-9218-466114b956e6 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:41.812839 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cab82b-da35-4d8c-9218-466114b956e6 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:41.812964 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cab82b-da35-4d8c-9218-466114b956e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:41.838880 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-82cab82b-da35-4d8c-9218-466114b956e6 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:28:41.840559 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-82cab82b-da35-4d8c-9218-466114b956e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:41.861227 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:28:41.914720 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1190/2344] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:28:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:28:41.924731 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4de192-d3ba-46f0-96a9-700a4d8e25aa None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:41.925243 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4de192-d3ba-46f0-96a9-700a4d8e25aa None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:41.925243 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4de192-d3ba-46f0-96a9-700a4d8e25aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:41.946107 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-0c4de192-d3ba-46f0-96a9-700a4d8e25aa None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:28:41.947222 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-0c4de192-d3ba-46f0-96a9-700a4d8e25aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:41.956124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1155/2345] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:28:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:28:41.960436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cab82b-da35-4d8c-9218-466114b956e6 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:41.972551 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cab82b-da35-4d8c-9218-466114b956e6 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:41.989867 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83c531-b87b-43cd-b722-0aced5deda8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:41.990498 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83c531-b87b-43cd-b722-0aced5deda8c admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:41.990769 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83c531-b87b-43cd-b722-0aced5deda8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:41.991076 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83c531-b87b-43cd-b722-0aced5deda8c admin admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:41.991903 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d83c531-b87b-43cd-b722-0aced5deda8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:28:41.992204 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d83c531-b87b-43cd-b722-0aced5deda8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:41.994178 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d83c531-b87b-43cd-b722-0aced5deda8c admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:42.013565 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1191/2346] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:28:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:28:42.014762 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4de192-d3ba-46f0-96a9-700a4d8e25aa None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:42.024488 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4de192-d3ba-46f0-96a9-700a4d8e25aa None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:42.029432 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae062a57-848a-4aee-bb82-f125e4e10287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:42.030063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae062a57-848a-4aee-bb82-f125e4e10287 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:42.030381 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae062a57-848a-4aee-bb82-f125e4e10287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:42.030601 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae062a57-848a-4aee-bb82-f125e4e10287 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:42.031266 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae062a57-848a-4aee-bb82-f125e4e10287 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:42.033721 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae062a57-848a-4aee-bb82-f125e4e10287 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:42.108451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1156/2347] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:28:42 2022] POST /identity/v3/projects => generated 377 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:28:42.112546 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83c531-b87b-43cd-b722-0aced5deda8c admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:42.123559 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83c531-b87b-43cd-b722-0aced5deda8c admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:42.194769 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aefe445-a890-401b-9eb2-08d4aa90f238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:42.198949 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aefe445-a890-401b-9eb2-08d4aa90f238 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:42.198949 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aefe445-a890-401b-9eb2-08d4aa90f238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:42.198949 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aefe445-a890-401b-9eb2-08d4aa90f238 admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:42.198949 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aefe445-a890-401b-9eb2-08d4aa90f238 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:42.199714 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aefe445-a890-401b-9eb2-08d4aa90f238 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:42.280636 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1192/2348] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:28:42 2022] POST /identity/v3/users => generated 289 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:28:42.285777 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae062a57-848a-4aee-bb82-f125e4e10287 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:42.297362 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae062a57-848a-4aee-bb82-f125e4e10287 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:42.307671 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d85e14-6ff8-49f2-955a-be9faf9ab9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:42.308966 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d85e14-6ff8-49f2-955a-be9faf9ab9ef admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:42.309432 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d85e14-6ff8-49f2-955a-be9faf9ab9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:42.310282 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d85e14-6ff8-49f2-955a-be9faf9ab9ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:42.311919 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d85e14-6ff8-49f2-955a-be9faf9ab9ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:42.315902 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d85e14-6ff8-49f2-955a-be9faf9ab9ef admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:42.337924 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1157/2349] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:28:42.342408 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aefe445-a890-401b-9eb2-08d4aa90f238 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:42.354390 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aefe445-a890-401b-9eb2-08d4aa90f238 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:42.359414 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be554a44-6f94-42e1-9fde-dc7954cd4fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:42.360040 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be554a44-6f94-42e1-9fde-dc7954cd4fd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:42.360316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be554a44-6f94-42e1-9fde-dc7954cd4fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:42.360778 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be554a44-6f94-42e1-9fde-dc7954cd4fd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:42.361651 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be554a44-6f94-42e1-9fde-dc7954cd4fd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:42.363622 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be554a44-6f94-42e1-9fde-dc7954cd4fd3 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:42.377587 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1193/2350] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:28:42.384674 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d85e14-6ff8-49f2-955a-be9faf9ab9ef admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:42.397940 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d85e14-6ff8-49f2-955a-be9faf9ab9ef admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:42.403244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425333ab-d3e8-4bf0-8718-8f590a18ab6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:42.403917 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425333ab-d3e8-4bf0-8718-8f590a18ab6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:42.404195 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425333ab-d3e8-4bf0-8718-8f590a18ab6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:42.404729 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425333ab-d3e8-4bf0-8718-8f590a18ab6d admin admin] PATH_INFO: `/v3/projects/371b57834ab642b5a00f28e272140f8d/users/fc067df40de64e8ebe80ba2652daebbb/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:42.424116 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425333ab-d3e8-4bf0-8718-8f590a18ab6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=371b57834ab642b5a00f28e272140f8d, user_id=fc067df40de64e8ebe80ba2652daebbb, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:42.426951 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425333ab-d3e8-4bf0-8718-8f590a18ab6d admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:42.532857 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1158/2351] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:28:42 2022] PUT /identity/v3/projects/371b57834ab642b5a00f28e272140f8d/users/fc067df40de64e8ebe80ba2652daebbb/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:28:42.536968 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee462533-1b96-4ad8-8ba0-a622183b68c1 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:42.537256 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee462533-1b96-4ad8-8ba0-a622183b68c1 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:42.537527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee462533-1b96-4ad8-8ba0-a622183b68c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:42.577599 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-ee462533-1b96-4ad8-8ba0-a622183b68c1 None None] MFA Rules not processed for user `fc067df40de64e8ebe80ba2652daebbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:28:42.578820 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-ee462533-1b96-4ad8-8ba0-a622183b68c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:42.653704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1194/2352] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:28:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:28:42.659245 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425333ab-d3e8-4bf0-8718-8f590a18ab6d admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:42.680207 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:28:42.740863 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425333ab-d3e8-4bf0-8718-8f590a18ab6d admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:42.746055 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd8c454-5e85-4047-ac09-65d9e3d53d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:42.746907 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd8c454-5e85-4047-ac09-65d9e3d53d2a admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:42.747048 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd8c454-5e85-4047-ac09-65d9e3d53d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:42.747744 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd8c454-5e85-4047-ac09-65d9e3d53d2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:42.747884 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd8c454-5e85-4047-ac09-65d9e3d53d2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:42.749833 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd8c454-5e85-4047-ac09-65d9e3d53d2a admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:42.870719 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1159/2353] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:28:42 2022] POST /identity/v3/projects => generated 377 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:28:42.872004 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee462533-1b96-4ad8-8ba0-a622183b68c1 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:42.883534 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee462533-1b96-4ad8-8ba0-a622183b68c1 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:42.895967 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:28:42.940047 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03320c-3e5b-47ab-92eb-7cc1a3bbf0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:42.940686 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03320c-3e5b-47ab-92eb-7cc1a3bbf0b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:42.940993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03320c-3e5b-47ab-92eb-7cc1a3bbf0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:42.941231 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03320c-3e5b-47ab-92eb-7cc1a3bbf0b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:42.941885 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb03320c-3e5b-47ab-92eb-7cc1a3bbf0b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:42.943792 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb03320c-3e5b-47ab-92eb-7cc1a3bbf0b7 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:42.999428 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1195/2354] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:28:42 2022] POST /identity/v3/users => generated 289 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:28:43.003867 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd8c454-5e85-4047-ac09-65d9e3d53d2a admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:43.014607 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd8c454-5e85-4047-ac09-65d9e3d53d2a admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:43.019801 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf62a1-3458-4011-ba0d-542d2b8cadc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:43.020399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf62a1-3458-4011-ba0d-542d2b8cadc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:43.020702 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf62a1-3458-4011-ba0d-542d2b8cadc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:43.020971 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf62a1-3458-4011-ba0d-542d2b8cadc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:43.021651 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdf62a1-3458-4011-ba0d-542d2b8cadc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:43.023226 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdf62a1-3458-4011-ba0d-542d2b8cadc2 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:43.034599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1160/2355] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:28:43.038689 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03320c-3e5b-47ab-92eb-7cc1a3bbf0b7 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:43.049329 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03320c-3e5b-47ab-92eb-7cc1a3bbf0b7 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:43.054078 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ea0aa8-3b5a-4041-91b5-267267d45842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:43.054806 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea0aa8-3b5a-4041-91b5-267267d45842 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:43.055514 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea0aa8-3b5a-4041-91b5-267267d45842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:43.055787 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea0aa8-3b5a-4041-91b5-267267d45842 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:43.056516 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ea0aa8-3b5a-4041-91b5-267267d45842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:43.058108 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ea0aa8-3b5a-4041-91b5-267267d45842 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:43.068730 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1196/2356] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:28:43.072714 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf62a1-3458-4011-ba0d-542d2b8cadc2 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:43.082561 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf62a1-3458-4011-ba0d-542d2b8cadc2 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:43.087466 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4038e6-09c6-47b6-a04a-68d459c941c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:43.088128 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4038e6-09c6-47b6-a04a-68d459c941c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:43.088384 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4038e6-09c6-47b6-a04a-68d459c941c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:43.088684 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4038e6-09c6-47b6-a04a-68d459c941c8 admin admin] PATH_INFO: `/v3/projects/ee2fb5f8c513481ab6e1299acc755004/users/2bcd3be1d76f4e14a34dc2642ca8930a/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:43.103780 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4038e6-09c6-47b6-a04a-68d459c941c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee2fb5f8c513481ab6e1299acc755004, user_id=2bcd3be1d76f4e14a34dc2642ca8930a, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:43.106428 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4038e6-09c6-47b6-a04a-68d459c941c8 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:43.193877 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1161/2357] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:28:43 2022] PUT /identity/v3/projects/ee2fb5f8c513481ab6e1299acc755004/users/2bcd3be1d76f4e14a34dc2642ca8930a/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:28:43.208670 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf273556-625c-467f-ad95-48e59f3dcaea None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:43.208952 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf273556-625c-467f-ad95-48e59f3dcaea None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:43.209255 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf273556-625c-467f-ad95-48e59f3dcaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:43.239749 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-bf273556-625c-467f-ad95-48e59f3dcaea None None] MFA Rules not processed for user `2bcd3be1d76f4e14a34dc2642ca8930a`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:28:43.240991 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-bf273556-625c-467f-ad95-48e59f3dcaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:43.311407 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1197/2358] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:28:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:28:43.313355 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4038e6-09c6-47b6-a04a-68d459c941c8 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:43.380475 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4038e6-09c6-47b6-a04a-68d459c941c8 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:43.385236 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef598c-8a13-4e23-9c6b-472aac2aebf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:43.385849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef598c-8a13-4e23-9c6b-472aac2aebf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:43.386130 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef598c-8a13-4e23-9c6b-472aac2aebf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:43.386407 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef598c-8a13-4e23-9c6b-472aac2aebf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:43.387079 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ef598c-8a13-4e23-9c6b-472aac2aebf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:43.389022 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ef598c-8a13-4e23-9c6b-472aac2aebf7 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:43.467842 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1162/2359] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:28:43 2022] POST /identity/v3/projects => generated 375 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:28:43.471962 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf273556-625c-467f-ad95-48e59f3dcaea None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:43.482587 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf273556-625c-467f-ad95-48e59f3dcaea None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:43.539665 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c02833-d2fb-4478-b775-c6f0f40e8b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:43.540282 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c02833-d2fb-4478-b775-c6f0f40e8b76 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:43.540590 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c02833-d2fb-4478-b775-c6f0f40e8b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:43.540860 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c02833-d2fb-4478-b775-c6f0f40e8b76 admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:43.541512 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c02833-d2fb-4478-b775-c6f0f40e8b76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:43.543407 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c02833-d2fb-4478-b775-c6f0f40e8b76 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:43.610245 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1198/2360] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:28:43 2022] POST /identity/v3/users => generated 294 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:28:43.614830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef598c-8a13-4e23-9c6b-472aac2aebf7 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:43.625062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef598c-8a13-4e23-9c6b-472aac2aebf7 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:43.632813 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac9a1ff-17d6-4216-898f-ee23097552cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:43.632813 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9a1ff-17d6-4216-898f-ee23097552cc admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:43.632813 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9a1ff-17d6-4216-898f-ee23097552cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:43.632813 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9a1ff-17d6-4216-898f-ee23097552cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:43.633262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac9a1ff-17d6-4216-898f-ee23097552cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:43.634913 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac9a1ff-17d6-4216-898f-ee23097552cc admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:43.647330 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1163/2361] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:28:43.651745 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c02833-d2fb-4478-b775-c6f0f40e8b76 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:43.663184 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c02833-d2fb-4478-b775-c6f0f40e8b76 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:43.674830 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85528827-bf7e-49de-a596-619096e94901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:43.675536 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85528827-bf7e-49de-a596-619096e94901 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:43.675866 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85528827-bf7e-49de-a596-619096e94901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:43.676424 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85528827-bf7e-49de-a596-619096e94901 admin admin] PATH_INFO: `/v3/projects/39ead751c5f94183b99bd72df22c7b84/users/e3d053e0ec1642b7b7ad111f88d22cae/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:43.691736 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:28:43.694932 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85528827-bf7e-49de-a596-619096e94901 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39ead751c5f94183b99bd72df22c7b84, user_id=e3d053e0ec1642b7b7ad111f88d22cae, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:43.697640 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85528827-bf7e-49de-a596-619096e94901 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:43.748072 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1199/2362] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:28:43 2022] PUT /identity/v3/projects/39ead751c5f94183b99bd72df22c7b84/users/e3d053e0ec1642b7b7ad111f88d22cae/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:28:43.752910 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac9a1ff-17d6-4216-898f-ee23097552cc admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:43.763196 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac9a1ff-17d6-4216-898f-ee23097552cc admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:43.805928 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:28:43.821296 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7162d-8f12-408e-8524-223ca067706f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:43.821922 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7162d-8f12-408e-8524-223ca067706f admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:43.822228 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7162d-8f12-408e-8524-223ca067706f admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:43.822501 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7162d-8f12-408e-8524-223ca067706f admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:43.823187 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7162d-8f12-408e-8524-223ca067706f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:43.824853 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7162d-8f12-408e-8524-223ca067706f admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:43.838208 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1164/2363] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:28:43.839631 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85528827-bf7e-49de-a596-619096e94901 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:43.851031 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85528827-bf7e-49de-a596-619096e94901 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:43.855746 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6bf37a-5210-4091-862b-45ffa4e751de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:43.856397 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6bf37a-5210-4091-862b-45ffa4e751de admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:43.856703 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6bf37a-5210-4091-862b-45ffa4e751de admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:43.857021 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6bf37a-5210-4091-862b-45ffa4e751de admin admin] PATH_INFO: `/v3/projects/39ead751c5f94183b99bd72df22c7b84/users/e3d053e0ec1642b7b7ad111f88d22cae/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:43.858970 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6bf37a-5210-4091-862b-45ffa4e751de admin admin] RBAC: Authorizing `identity:create_grant(project_id=39ead751c5f94183b99bd72df22c7b84, user_id=e3d053e0ec1642b7b7ad111f88d22cae, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:43.862814 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6bf37a-5210-4091-862b-45ffa4e751de admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:43.872938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1200/2364] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:28:43 2022] PUT /identity/v3/projects/39ead751c5f94183b99bd72df22c7b84/users/e3d053e0ec1642b7b7ad111f88d22cae/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:28:43.884172 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9794150a-48e0-43cf-b02b-5648830c4c92 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:43.884479 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9794150a-48e0-43cf-b02b-5648830c4c92 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:43.884750 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9794150a-48e0-43cf-b02b-5648830c4c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:43.914018 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-9794150a-48e0-43cf-b02b-5648830c4c92 None None] MFA Rules not processed for user `e3d053e0ec1642b7b7ad111f88d22cae`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:28:43.915214 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-9794150a-48e0-43cf-b02b-5648830c4c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:43.982271 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1165/2365] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:28:43 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:28:43.994461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6bf37a-5210-4091-862b-45ffa4e751de admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:43.996126 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-2e6bf37a-5210-4091-862b-45ffa4e751de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:44.054205 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6bf37a-5210-4091-862b-45ffa4e751de admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:44.057901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f8c78-a8c2-4b77-972c-e677af50057a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:44.058441 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f8c78-a8c2-4b77-972c-e677af50057a service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:44.058714 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f8c78-a8c2-4b77-972c-e677af50057a service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:44.058983 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f8c78-a8c2-4b77-972c-e677af50057a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:44.114237 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f8c78-a8c2-4b77-972c-e677af50057a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:44.115669 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f8c78-a8c2-4b77-972c-e677af50057a service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:44.124387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1201/2366] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:28:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:28:47.128638 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9794150a-48e0-43cf-b02b-5648830c4c92 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:47.130033 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-9794150a-48e0-43cf-b02b-5648830c4c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:47.142097 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9794150a-48e0-43cf-b02b-5648830c4c92 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:47.145726 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070e093-9da0-49bb-9ca5-ba8571cd1859 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:47.146340 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070e093-9da0-49bb-9ca5-ba8571cd1859 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:47.146615 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070e093-9da0-49bb-9ca5-ba8571cd1859 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:47.146885 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070e093-9da0-49bb-9ca5-ba8571cd1859 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:47.204641 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4070e093-9da0-49bb-9ca5-ba8571cd1859 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:47.206070 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4070e093-9da0-49bb-9ca5-ba8571cd1859 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:47.215144 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1166/2367] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:28:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:28:48.291733 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f8c78-a8c2-4b77-972c-e677af50057a service cinder] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:48.293183 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-cf7f8c78-a8c2-4b77-972c-e677af50057a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:48.366880 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f8c78-a8c2-4b77-972c-e677af50057a service cinder] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:48.371974 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7985b5-7d7f-48c4-80c1-ad988e923552 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff761f6f81a46098f94bd2ec2be9c1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:48.372624 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7985b5-7d7f-48c4-80c1-ad988e923552 service placement] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:48.372902 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7985b5-7d7f-48c4-80c1-ad988e923552 service placement] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:48.373175 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7985b5-7d7f-48c4-80c1-ad988e923552 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:48.374955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-8d7985b5-7d7f-48c4-80c1-ad988e923552 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:48.387200 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7985b5-7d7f-48c4-80c1-ad988e923552 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:48.388886 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7985b5-7d7f-48c4-80c1-ad988e923552 service placement] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:48.399590 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1202/2368] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:28:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:28:48.663659 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070e093-9da0-49bb-9ca5-ba8571cd1859 service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:48.672358 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070e093-9da0-49bb-9ca5-ba8571cd1859 service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:48.676104 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016a826-e90d-4dab-8541-5be0cda9b9b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:48.676803 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016a826-e90d-4dab-8541-5be0cda9b9b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:48.677079 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016a826-e90d-4dab-8541-5be0cda9b9b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:48.677345 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016a826-e90d-4dab-8541-5be0cda9b9b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:48.686536 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e016a826-e90d-4dab-8541-5be0cda9b9b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:48.687945 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e016a826-e90d-4dab-8541-5be0cda9b9b3 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:48.698087 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1167/2369] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:28:48 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:28:48.839749 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7985b5-7d7f-48c4-80c1-ad988e923552 service placement] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:48.841193 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-8d7985b5-7d7f-48c4-80c1-ad988e923552 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:48.869220 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:28:48.912799 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7985b5-7d7f-48c4-80c1-ad988e923552 service placement] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:48.917765 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e687807-6ea6-4f6a-a807-5733ae7d9600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:48.920549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e687807-6ea6-4f6a-a807-5733ae7d9600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:48.920549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e687807-6ea6-4f6a-a807-5733ae7d9600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:48.920549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e687807-6ea6-4f6a-a807-5733ae7d9600 admin admin] PATH_INFO: `/v3/users/f894c4143c754f1987ba0e82f6bc390b` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:48.920549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e687807-6ea6-4f6a-a807-5733ae7d9600 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f894c4143c754f1987ba0e82f6bc390b)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:48.922034 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e687807-6ea6-4f6a-a807-5733ae7d9600 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:49.148518 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-6e687807-6ea6-4f6a-a807-5733ae7d9600 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f894c4143c754f1987ba0e82f6bc390b'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:49.155555 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-6e687807-6ea6-4f6a-a807-5733ae7d9600 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f894c4143c754f1987ba0e82f6bc390b'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:49.161968 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-6e687807-6ea6-4f6a-a807-5733ae7d9600 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f894c4143c754f1987ba0e82f6bc390b'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:49.161968 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-6e687807-6ea6-4f6a-a807-5733ae7d9600 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f894c4143c754f1987ba0e82f6bc390b'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:49.169127 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-6e687807-6ea6-4f6a-a807-5733ae7d9600 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f894c4143c754f1987ba0e82f6bc390b'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:49.243940 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1203/2370] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:28:48 2022] DELETE /identity/v3/users/f894c4143c754f1987ba0e82f6bc390b => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:28:49.248622 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f2d2b-b262-4120-aa04-0b6d11f61e3b None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:49.248912 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f2d2b-b262-4120-aa04-0b6d11f61e3b None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:49.249239 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f2d2b-b262-4120-aa04-0b6d11f61e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:49.268023 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-695f2d2b-b262-4120-aa04-0b6d11f61e3b None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:28:49.269285 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-695f2d2b-b262-4120-aa04-0b6d11f61e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:49.317330 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:28:49.332899 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1168/2371] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:28:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:28:49.342202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e687807-6ea6-4f6a-a807-5733ae7d9600 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:49.343668 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-6e687807-6ea6-4f6a-a807-5733ae7d9600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:49.402224 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e687807-6ea6-4f6a-a807-5733ae7d9600 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:49.406558 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae482a6-b96d-4ff0-9ebd-4bc0fe5e8b43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:49.407175 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae482a6-b96d-4ff0-9ebd-4bc0fe5e8b43 service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:49.407453 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae482a6-b96d-4ff0-9ebd-4bc0fe5e8b43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:49.407725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae482a6-b96d-4ff0-9ebd-4bc0fe5e8b43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:49.418428 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae482a6-b96d-4ff0-9ebd-4bc0fe5e8b43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:49.420544 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae482a6-b96d-4ff0-9ebd-4bc0fe5e8b43 service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:49.430724 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1204/2372] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:28:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:28:49.915039 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f2d2b-b262-4120-aa04-0b6d11f61e3b None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:49.917045 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-695f2d2b-b262-4120-aa04-0b6d11f61e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:49.929332 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f2d2b-b262-4120-aa04-0b6d11f61e3b None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:49.934800 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5951a-3663-479d-946b-f6c6f61c88e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:49.935452 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5951a-3663-479d-946b-f6c6f61c88e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:49.936580 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5951a-3663-479d-946b-f6c6f61c88e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:49.936580 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5951a-3663-479d-946b-f6c6f61c88e2 admin admin] PATH_INFO: `/v3/projects/6d19c4cd8cf94027b6d9c1acba0e549a` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:49.937244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e5951a-3663-479d-946b-f6c6f61c88e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d19c4cd8cf94027b6d9c1acba0e549a)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:49.939141 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e5951a-3663-479d-946b-f6c6f61c88e2 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:50.050639 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e5e5951a-3663-479d-946b-f6c6f61c88e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d19c4cd8cf94027b6d9c1acba0e549a', 'request_id': 'req-e5e5951a-3663-479d-946b-f6c6f61c88e2'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:50.058375 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e5e5951a-3663-479d-946b-f6c6f61c88e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d19c4cd8cf94027b6d9c1acba0e549a', 'request_id': 'req-e5e5951a-3663-479d-946b-f6c6f61c88e2'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:50.130690 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e5e5951a-3663-479d-946b-f6c6f61c88e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d19c4cd8cf94027b6d9c1acba0e549a', 'request_id': 'req-e5e5951a-3663-479d-946b-f6c6f61c88e2'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:50.142362 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e5e5951a-3663-479d-946b-f6c6f61c88e2 admin admin] The token cache is being invalidate because project 6d19c4cd8cf94027b6d9c1acba0e549a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:28:50.142646 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e5e5951a-3663-479d-946b-f6c6f61c88e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:50.143288 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e5e5951a-3663-479d-946b-f6c6f61c88e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:50.144966 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1169/2373] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:28:49 2022] DELETE /identity/v3/projects/6d19c4cd8cf94027b6d9c1acba0e549a => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:28:50.149820 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4fe529-766f-4e57-8d3f-17e4f6d7f739 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:50.150172 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4fe529-766f-4e57-8d3f-17e4f6d7f739 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:50.150515 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4fe529-766f-4e57-8d3f-17e4f6d7f739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:50.171019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-8e4fe529-766f-4e57-8d3f-17e4f6d7f739 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:28:50.172224 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-8e4fe529-766f-4e57-8d3f-17e4f6d7f739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:50.224190 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:28:50.244783 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1205/2374] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:28:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:28:50.252506 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6727393-807d-4426-aeed-7db6a72d53e9 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:50.252802 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6727393-807d-4426-aeed-7db6a72d53e9 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:50.253076 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6727393-807d-4426-aeed-7db6a72d53e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:50.273094 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-c6727393-807d-4426-aeed-7db6a72d53e9 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:28:50.274293 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c6727393-807d-4426-aeed-7db6a72d53e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:50.282263 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1170/2375] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:28:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:28:50.288328 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4fe529-766f-4e57-8d3f-17e4f6d7f739 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:50.298184 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4fe529-766f-4e57-8d3f-17e4f6d7f739 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:50.303370 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230091f7-5e30-4b0c-8eca-9e8da44f7ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:50.304046 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230091f7-5e30-4b0c-8eca-9e8da44f7ad2 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:50.304451 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230091f7-5e30-4b0c-8eca-9e8da44f7ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:50.304771 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230091f7-5e30-4b0c-8eca-9e8da44f7ad2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:50.305598 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230091f7-5e30-4b0c-8eca-9e8da44f7ad2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:28:50.305896 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230091f7-5e30-4b0c-8eca-9e8da44f7ad2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:50.307778 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230091f7-5e30-4b0c-8eca-9e8da44f7ad2 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:50.321338 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1206/2376] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:28:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:28:50.325948 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6727393-807d-4426-aeed-7db6a72d53e9 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:50.338210 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6727393-807d-4426-aeed-7db6a72d53e9 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:50.343813 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a169b85-8883-45b3-b0f5-715265a0cba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:50.344488 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a169b85-8883-45b3-b0f5-715265a0cba7 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:50.344766 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a169b85-8883-45b3-b0f5-715265a0cba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:50.345162 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a169b85-8883-45b3-b0f5-715265a0cba7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:50.345900 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a169b85-8883-45b3-b0f5-715265a0cba7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:50.347930 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a169b85-8883-45b3-b0f5-715265a0cba7 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:50.445707 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1171/2377] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:28:50 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:28:50.450188 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230091f7-5e30-4b0c-8eca-9e8da44f7ad2 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:50.462160 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230091f7-5e30-4b0c-8eca-9e8da44f7ad2 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:50.543104 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a71450-1475-4f1c-99c6-79ea2fcf839e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:50.543918 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a71450-1475-4f1c-99c6-79ea2fcf839e admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:50.544201 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a71450-1475-4f1c-99c6-79ea2fcf839e admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:50.544629 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a71450-1475-4f1c-99c6-79ea2fcf839e admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:50.545296 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a71450-1475-4f1c-99c6-79ea2fcf839e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:50.547206 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a71450-1475-4f1c-99c6-79ea2fcf839e admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:50.638267 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1207/2378] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:28:50 2022] POST /identity/v3/users => generated 292 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:28:50.642375 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a169b85-8883-45b3-b0f5-715265a0cba7 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:50.652854 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a169b85-8883-45b3-b0f5-715265a0cba7 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:50.657729 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab546c21-055a-4a4a-821a-e3feed2c842a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:50.658344 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab546c21-055a-4a4a-821a-e3feed2c842a admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:50.658623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab546c21-055a-4a4a-821a-e3feed2c842a admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:50.658894 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab546c21-055a-4a4a-821a-e3feed2c842a admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:50.660176 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab546c21-055a-4a4a-821a-e3feed2c842a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:50.662247 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab546c21-055a-4a4a-821a-e3feed2c842a admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:50.672424 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1172/2379] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:28:50.677096 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a71450-1475-4f1c-99c6-79ea2fcf839e admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:50.692507 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a71450-1475-4f1c-99c6-79ea2fcf839e admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:50.697173 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60076570-f6a3-47f2-bb1b-7ce8ac7b0800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:50.697776 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60076570-f6a3-47f2-bb1b-7ce8ac7b0800 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:50.698081 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60076570-f6a3-47f2-bb1b-7ce8ac7b0800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:50.698360 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60076570-f6a3-47f2-bb1b-7ce8ac7b0800 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:50.699045 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60076570-f6a3-47f2-bb1b-7ce8ac7b0800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:50.700691 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60076570-f6a3-47f2-bb1b-7ce8ac7b0800 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:50.715652 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1208/2380] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:28:50.716869 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab546c21-055a-4a4a-821a-e3feed2c842a admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:50.728679 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab546c21-055a-4a4a-821a-e3feed2c842a admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:50.734567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a48227-3855-4b23-bee8-6b0ce407c732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:50.775876 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a48227-3855-4b23-bee8-6b0ce407c732 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:50.775876 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a48227-3855-4b23-bee8-6b0ce407c732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:50.775876 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a48227-3855-4b23-bee8-6b0ce407c732 admin admin] PATH_INFO: `/v3/projects/5e50310a4bcd4dd5acde809956702d0b/users/cee6d9ed0c344584b8bf2fbe3c4178bf/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:50.775876 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a48227-3855-4b23-bee8-6b0ce407c732 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e50310a4bcd4dd5acde809956702d0b, user_id=cee6d9ed0c344584b8bf2fbe3c4178bf, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:50.775876 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a48227-3855-4b23-bee8-6b0ce407c732 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:50.879099 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1173/2381] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:28:50 2022] PUT /identity/v3/projects/5e50310a4bcd4dd5acde809956702d0b/users/cee6d9ed0c344584b8bf2fbe3c4178bf/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:28:50.884197 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a4ec8-65c0-41d4-8165-3f96aa17e60f None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:50.888469 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a4ec8-65c0-41d4-8165-3f96aa17e60f None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:50.888949 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a4ec8-65c0-41d4-8165-3f96aa17e60f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:50.911497 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-721a4ec8-65c0-41d4-8165-3f96aa17e60f None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:28:50.912767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-721a4ec8-65c0-41d4-8165-3f96aa17e60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:50.984963 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1209/2382] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:28:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:28:50.995557 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a48227-3855-4b23-bee8-6b0ce407c732 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:50.997375 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-b4a48227-3855-4b23-bee8-6b0ce407c732 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:51.089959 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a48227-3855-4b23-bee8-6b0ce407c732 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:51.099869 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956303e7-75e1-4415-9a16-cd67d269817b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:51.100609 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956303e7-75e1-4415-9a16-cd67d269817b service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:51.100898 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956303e7-75e1-4415-9a16-cd67d269817b service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:51.101160 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956303e7-75e1-4415-9a16-cd67d269817b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:51.114375 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956303e7-75e1-4415-9a16-cd67d269817b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:51.115814 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956303e7-75e1-4415-9a16-cd67d269817b service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:51.126541 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1174/2383] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:28:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:28:52.751010 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1010f06-702d-4d32-9316-3c3bb4c17c60 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:52.751473 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1010f06-702d-4d32-9316-3c3bb4c17c60 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:52.751801 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1010f06-702d-4d32-9316-3c3bb4c17c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:52.794300 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-d1010f06-702d-4d32-9316-3c3bb4c17c60 None None] MFA Rules not processed for user `cee6d9ed0c344584b8bf2fbe3c4178bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:28:52.795617 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-d1010f06-702d-4d32-9316-3c3bb4c17c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:52.842893 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:28:52.871578 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1210/2384] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:28:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:28:52.889076 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956303e7-75e1-4415-9a16-cd67d269817b service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:52.889076 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-956303e7-75e1-4415-9a16-cd67d269817b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:52.904660 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:28:52.971271 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956303e7-75e1-4415-9a16-cd67d269817b service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:52.983989 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b21d9e-b930-43d3-a2b2-b79adb6f96b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:52.985117 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b21d9e-b930-43d3-a2b2-b79adb6f96b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:52.985594 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b21d9e-b930-43d3-a2b2-b79adb6f96b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:52.985878 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b21d9e-b930-43d3-a2b2-b79adb6f96b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:52.997417 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b21d9e-b930-43d3-a2b2-b79adb6f96b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:53.002128 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b21d9e-b930-43d3-a2b2-b79adb6f96b1 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:53.032225 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1175/2385] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:28:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:28:55.029439 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1010f06-702d-4d32-9316-3c3bb4c17c60 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:55.030984 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-d1010f06-702d-4d32-9316-3c3bb4c17c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:55.048169 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1010f06-702d-4d32-9316-3c3bb4c17c60 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:55.054729 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d2cee-613d-42c1-b1d9-840688d9ae3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:55.061148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d2cee-613d-42c1-b1d9-840688d9ae3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:55.061148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d2cee-613d-42c1-b1d9-840688d9ae3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:55.061148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d2cee-613d-42c1-b1d9-840688d9ae3e admin admin] PATH_INFO: `/v3/users/fc067df40de64e8ebe80ba2652daebbb` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:55.061148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63d2cee-613d-42c1-b1d9-840688d9ae3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc067df40de64e8ebe80ba2652daebbb)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:55.061148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63d2cee-613d-42c1-b1d9-840688d9ae3e admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:55.225100 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-e63d2cee-613d-42c1-b1d9-840688d9ae3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc067df40de64e8ebe80ba2652daebbb'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:55.231922 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-e63d2cee-613d-42c1-b1d9-840688d9ae3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc067df40de64e8ebe80ba2652daebbb'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:55.232787 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-e63d2cee-613d-42c1-b1d9-840688d9ae3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc067df40de64e8ebe80ba2652daebbb'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:55.233481 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-e63d2cee-613d-42c1-b1d9-840688d9ae3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc067df40de64e8ebe80ba2652daebbb'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:55.249185 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-e63d2cee-613d-42c1-b1d9-840688d9ae3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc067df40de64e8ebe80ba2652daebbb'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:55.346135 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1211/2386] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:28:55 2022] DELETE /identity/v3/users/fc067df40de64e8ebe80ba2652daebbb => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:28:55.350653 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab60a71-7578-4c1e-9c57-f4edb654ddb8 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:55.351110 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab60a71-7578-4c1e-9c57-f4edb654ddb8 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:55.351227 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab60a71-7578-4c1e-9c57-f4edb654ddb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:55.370017 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-9ab60a71-7578-4c1e-9c57-f4edb654ddb8 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:28:55.371364 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-9ab60a71-7578-4c1e-9c57-f4edb654ddb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:55.430656 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1176/2387] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:28:55.440479 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d2cee-613d-42c1-b1d9-840688d9ae3e admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:55.442629 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-e63d2cee-613d-42c1-b1d9-840688d9ae3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:55.497827 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d2cee-613d-42c1-b1d9-840688d9ae3e admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:55.501348 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd2c5c3-849f-4bb8-bad4-9b55d3be846e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:55.501954 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2c5c3-849f-4bb8-bad4-9b55d3be846e service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:55.502230 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2c5c3-849f-4bb8-bad4-9b55d3be846e service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:55.502501 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2c5c3-849f-4bb8-bad4-9b55d3be846e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:55.513615 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd2c5c3-849f-4bb8-bad4-9b55d3be846e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:55.515045 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd2c5c3-849f-4bb8-bad4-9b55d3be846e service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:55.524942 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1212/2388] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:28:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:28:55.950748 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab60a71-7578-4c1e-9c57-f4edb654ddb8 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:55.952387 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-9ab60a71-7578-4c1e-9c57-f4edb654ddb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:55.956142 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823c6e0-d4a5-4dab-83c0-b982d2ed15ab None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:55.956499 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823c6e0-d4a5-4dab-83c0-b982d2ed15ab None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:55.957174 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823c6e0-d4a5-4dab-83c0-b982d2ed15ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:55.981908 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-d823c6e0-d4a5-4dab-83c0-b982d2ed15ab None None] MFA Rules not processed for user `cee6d9ed0c344584b8bf2fbe3c4178bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:28:55.982405 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab60a71-7578-4c1e-9c57-f4edb654ddb8 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:55.982622 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-d823c6e0-d4a5-4dab-83c0-b982d2ed15ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:55.984792 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9ec92-bb6a-4ee9-9288-d6a9ce2b5103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:55.985429 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9ec92-bb6a-4ee9-9288-d6a9ce2b5103 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:55.985704 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9ec92-bb6a-4ee9-9288-d6a9ce2b5103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:55.985974 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9ec92-bb6a-4ee9-9288-d6a9ce2b5103 admin admin] PATH_INFO: `/v3/projects/371b57834ab642b5a00f28e272140f8d` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:55.987419 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b9ec92-bb6a-4ee9-9288-d6a9ce2b5103 admin admin] RBAC: Authorizing `identity:delete_project(project_id=371b57834ab642b5a00f28e272140f8d)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:55.990159 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b9ec92-bb6a-4ee9-9288-d6a9ce2b5103 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:56.032080 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:28:56.041189 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1213/2389] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:28:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:28:56.093802 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f5b9ec92-bb6a-4ee9-9288-d6a9ce2b5103 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '371b57834ab642b5a00f28e272140f8d', 'request_id': 'req-f5b9ec92-bb6a-4ee9-9288-d6a9ce2b5103'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:56.101831 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f5b9ec92-bb6a-4ee9-9288-d6a9ce2b5103 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '371b57834ab642b5a00f28e272140f8d', 'request_id': 'req-f5b9ec92-bb6a-4ee9-9288-d6a9ce2b5103'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:56.180861 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f5b9ec92-bb6a-4ee9-9288-d6a9ce2b5103 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '371b57834ab642b5a00f28e272140f8d', 'request_id': 'req-f5b9ec92-bb6a-4ee9-9288-d6a9ce2b5103'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:56.193069 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f5b9ec92-bb6a-4ee9-9288-d6a9ce2b5103 admin admin] The token cache is being invalidate because project 371b57834ab642b5a00f28e272140f8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:28:56.193362 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f5b9ec92-bb6a-4ee9-9288-d6a9ce2b5103 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:56.194023 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f5b9ec92-bb6a-4ee9-9288-d6a9ce2b5103 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:56.195564 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1177/2390] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:28:55 2022] DELETE /identity/v3/projects/371b57834ab642b5a00f28e272140f8d => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:28:56.199504 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823c6e0-d4a5-4dab-83c0-b982d2ed15ab None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:56.211069 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823c6e0-d4a5-4dab-83c0-b982d2ed15ab None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:56.279038 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b859e-0b80-44b1-92c2-687f845c792b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:56.279770 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b859e-0b80-44b1-92c2-687f845c792b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:56.280151 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b859e-0b80-44b1-92c2-687f845c792b admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:56.280525 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b859e-0b80-44b1-92c2-687f845c792b admin admin] PATH_INFO: `/v3/users/2bcd3be1d76f4e14a34dc2642ca8930a` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:56.281877 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783b859e-0b80-44b1-92c2-687f845c792b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bcd3be1d76f4e14a34dc2642ca8930a)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:56.284456 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783b859e-0b80-44b1-92c2-687f845c792b admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:56.458778 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-783b859e-0b80-44b1-92c2-687f845c792b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bcd3be1d76f4e14a34dc2642ca8930a'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:56.465649 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-783b859e-0b80-44b1-92c2-687f845c792b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bcd3be1d76f4e14a34dc2642ca8930a'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:56.466421 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-783b859e-0b80-44b1-92c2-687f845c792b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bcd3be1d76f4e14a34dc2642ca8930a'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:56.467093 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-783b859e-0b80-44b1-92c2-687f845c792b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bcd3be1d76f4e14a34dc2642ca8930a'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:56.479608 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-783b859e-0b80-44b1-92c2-687f845c792b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bcd3be1d76f4e14a34dc2642ca8930a'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:56.589591 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1214/2391] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:28:56 2022] DELETE /identity/v3/users/2bcd3be1d76f4e14a34dc2642ca8930a => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:28:57.063125 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9ec92-bb6a-4ee9-9288-d6a9ce2b5103 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:57.073122 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f5b9ec92-bb6a-4ee9-9288-d6a9ce2b5103 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:57.078147 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9ec92-bb6a-4ee9-9288-d6a9ce2b5103 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:57.131675 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:28:57.149682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79074c0d-b725-47ff-8d7d-5f81612e33dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:57.150321 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79074c0d-b725-47ff-8d7d-5f81612e33dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:57.150597 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79074c0d-b725-47ff-8d7d-5f81612e33dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:57.150867 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79074c0d-b725-47ff-8d7d-5f81612e33dc admin admin] PATH_INFO: `/v3/projects/ee2fb5f8c513481ab6e1299acc755004` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:57.152005 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79074c0d-b725-47ff-8d7d-5f81612e33dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee2fb5f8c513481ab6e1299acc755004)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:57.153924 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79074c0d-b725-47ff-8d7d-5f81612e33dc admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:57.278953 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-79074c0d-b725-47ff-8d7d-5f81612e33dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee2fb5f8c513481ab6e1299acc755004', 'request_id': 'req-79074c0d-b725-47ff-8d7d-5f81612e33dc'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:57.285367 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-79074c0d-b725-47ff-8d7d-5f81612e33dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee2fb5f8c513481ab6e1299acc755004', 'request_id': 'req-79074c0d-b725-47ff-8d7d-5f81612e33dc'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:57.375346 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-79074c0d-b725-47ff-8d7d-5f81612e33dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee2fb5f8c513481ab6e1299acc755004', 'request_id': 'req-79074c0d-b725-47ff-8d7d-5f81612e33dc'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:57.386560 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-79074c0d-b725-47ff-8d7d-5f81612e33dc admin admin] The token cache is being invalidate because project ee2fb5f8c513481ab6e1299acc755004 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:28:57.388829 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-79074c0d-b725-47ff-8d7d-5f81612e33dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:57.388829 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-79074c0d-b725-47ff-8d7d-5f81612e33dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:57.389111 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1178/2392] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:28:57 2022] DELETE /identity/v3/projects/ee2fb5f8c513481ab6e1299acc755004 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:28:57.393080 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b859e-0b80-44b1-92c2-687f845c792b admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:57.394484 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-783b859e-0b80-44b1-92c2-687f845c792b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:57.407182 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b859e-0b80-44b1-92c2-687f845c792b admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:57.466842 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b25547c-81f1-4a3f-a103-89284f2fa1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:57.467490 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b25547c-81f1-4a3f-a103-89284f2fa1f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:57.467843 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b25547c-81f1-4a3f-a103-89284f2fa1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:57.468142 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b25547c-81f1-4a3f-a103-89284f2fa1f9 admin admin] PATH_INFO: `/v3/users/e3d053e0ec1642b7b7ad111f88d22cae` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:57.469333 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b25547c-81f1-4a3f-a103-89284f2fa1f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3d053e0ec1642b7b7ad111f88d22cae)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:57.471249 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b25547c-81f1-4a3f-a103-89284f2fa1f9 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:57.501609 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:28:57.714360 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-6b25547c-81f1-4a3f-a103-89284f2fa1f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3d053e0ec1642b7b7ad111f88d22cae'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:57.721991 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-6b25547c-81f1-4a3f-a103-89284f2fa1f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3d053e0ec1642b7b7ad111f88d22cae'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:57.722793 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-6b25547c-81f1-4a3f-a103-89284f2fa1f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3d053e0ec1642b7b7ad111f88d22cae'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:57.724046 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-6b25547c-81f1-4a3f-a103-89284f2fa1f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3d053e0ec1642b7b7ad111f88d22cae'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:57.736148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-6b25547c-81f1-4a3f-a103-89284f2fa1f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3d053e0ec1642b7b7ad111f88d22cae'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:57.833737 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1215/2393] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:28:57 2022] DELETE /identity/v3/users/e3d053e0ec1642b7b7ad111f88d22cae => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:28:58.320845 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79074c0d-b725-47ff-8d7d-5f81612e33dc admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:58.322253 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-79074c0d-b725-47ff-8d7d-5f81612e33dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:58.342434 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79074c0d-b725-47ff-8d7d-5f81612e33dc admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:58.401329 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c50ad7c-d35d-499f-8e5b-b04a942b013d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:58.401964 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c50ad7c-d35d-499f-8e5b-b04a942b013d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:58.402237 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c50ad7c-d35d-499f-8e5b-b04a942b013d admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:58.402507 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c50ad7c-d35d-499f-8e5b-b04a942b013d admin admin] PATH_INFO: `/v3/projects/39ead751c5f94183b99bd72df22c7b84` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:58.403659 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c50ad7c-d35d-499f-8e5b-b04a942b013d admin admin] RBAC: Authorizing `identity:delete_project(project_id=39ead751c5f94183b99bd72df22c7b84)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:58.405599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c50ad7c-d35d-499f-8e5b-b04a942b013d admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:58.526455 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8c50ad7c-d35d-499f-8e5b-b04a942b013d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39ead751c5f94183b99bd72df22c7b84', 'request_id': 'req-8c50ad7c-d35d-499f-8e5b-b04a942b013d'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:58.532774 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8c50ad7c-d35d-499f-8e5b-b04a942b013d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39ead751c5f94183b99bd72df22c7b84', 'request_id': 'req-8c50ad7c-d35d-499f-8e5b-b04a942b013d'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:58.628369 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8c50ad7c-d35d-499f-8e5b-b04a942b013d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39ead751c5f94183b99bd72df22c7b84', 'request_id': 'req-8c50ad7c-d35d-499f-8e5b-b04a942b013d'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:58.636784 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:28:58.640777 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8c50ad7c-d35d-499f-8e5b-b04a942b013d admin admin] The token cache is being invalidate because project 39ead751c5f94183b99bd72df22c7b84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:28:58.641068 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8c50ad7c-d35d-499f-8e5b-b04a942b013d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:58.641708 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8c50ad7c-d35d-499f-8e5b-b04a942b013d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:28:58.643215 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1179/2394] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:28:58 2022] DELETE /identity/v3/projects/39ead751c5f94183b99bd72df22c7b84 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:28:58.649973 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2833f4-a096-4846-9ab6-c0f042caf280 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:58.650482 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2833f4-a096-4846-9ab6-c0f042caf280 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:58.650597 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2833f4-a096-4846-9ab6-c0f042caf280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:58.671664 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-8b2833f4-a096-4846-9ab6-c0f042caf280 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:28:58.673525 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-8b2833f4-a096-4846-9ab6-c0f042caf280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:58.735690 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1216/2395] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:28:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:28:58.744343 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a746d8-838b-403a-8ee0-de37bfafe9b4 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:58.745016 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a746d8-838b-403a-8ee0-de37bfafe9b4 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:58.745516 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a746d8-838b-403a-8ee0-de37bfafe9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:58.767498 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-e5a746d8-838b-403a-8ee0-de37bfafe9b4 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:28:58.768756 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e5a746d8-838b-403a-8ee0-de37bfafe9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:58.780145 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1180/2396] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:28:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:28:58.782179 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2833f4-a096-4846-9ab6-c0f042caf280 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:58.792982 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2833f4-a096-4846-9ab6-c0f042caf280 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:58.797669 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d56c9a-0c89-4a02-a564-e154c633f07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:58.798464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d56c9a-0c89-4a02-a564-e154c633f07d admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:58.798707 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d56c9a-0c89-4a02-a564-e154c633f07d admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:58.798975 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d56c9a-0c89-4a02-a564-e154c633f07d admin admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:58.799750 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d56c9a-0c89-4a02-a564-e154c633f07d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:28:58.800149 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d56c9a-0c89-4a02-a564-e154c633f07d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:58.802342 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d56c9a-0c89-4a02-a564-e154c633f07d admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:58.814359 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1217/2397] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:28:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:28:58.820284 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a746d8-838b-403a-8ee0-de37bfafe9b4 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:58.830050 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a746d8-838b-403a-8ee0-de37bfafe9b4 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:58.835672 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2100bf0-e3e6-4f1e-b52f-7b9e9d193d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:58.836922 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2100bf0-e3e6-4f1e-b52f-7b9e9d193d65 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:58.837241 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2100bf0-e3e6-4f1e-b52f-7b9e9d193d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:58.837677 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2100bf0-e3e6-4f1e-b52f-7b9e9d193d65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:58.838415 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2100bf0-e3e6-4f1e-b52f-7b9e9d193d65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:58.840392 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2100bf0-e3e6-4f1e-b52f-7b9e9d193d65 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:58.928106 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1181/2398] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:28:58 2022] POST /identity/v3/projects => generated 397 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:28:58.932353 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d56c9a-0c89-4a02-a564-e154c633f07d admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:58.951371 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d56c9a-0c89-4a02-a564-e154c633f07d admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:59.032619 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3ee48b-46aa-40a5-8496-011de8c8fb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:59.033240 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ee48b-46aa-40a5-8496-011de8c8fb23 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:59.033524 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ee48b-46aa-40a5-8496-011de8c8fb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:59.033793 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ee48b-46aa-40a5-8496-011de8c8fb23 admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:59.034486 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3ee48b-46aa-40a5-8496-011de8c8fb23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:59.037074 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3ee48b-46aa-40a5-8496-011de8c8fb23 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:59.160110 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1218/2399] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:28:58 2022] POST /identity/v3/users => generated 299 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:28:59.165194 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2100bf0-e3e6-4f1e-b52f-7b9e9d193d65 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:59.175164 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2100bf0-e3e6-4f1e-b52f-7b9e9d193d65 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:59.180342 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62597662-25b3-411c-8f36-5bbc4af8ec71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:59.180973 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62597662-25b3-411c-8f36-5bbc4af8ec71 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:59.181247 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62597662-25b3-411c-8f36-5bbc4af8ec71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:59.181514 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62597662-25b3-411c-8f36-5bbc4af8ec71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:59.182246 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62597662-25b3-411c-8f36-5bbc4af8ec71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:59.183825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62597662-25b3-411c-8f36-5bbc4af8ec71 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:59.194331 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1182/2400] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:28:59.198253 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3ee48b-46aa-40a5-8496-011de8c8fb23 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:59.210395 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3ee48b-46aa-40a5-8496-011de8c8fb23 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:59.215791 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0670e6-23cf-4d71-9207-996a4af24e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:59.216439 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0670e6-23cf-4d71-9207-996a4af24e3c admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:59.216736 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0670e6-23cf-4d71-9207-996a4af24e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:59.216997 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0670e6-23cf-4d71-9207-996a4af24e3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:59.217687 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0670e6-23cf-4d71-9207-996a4af24e3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:59.219286 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0670e6-23cf-4d71-9207-996a4af24e3c admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:59.230988 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1219/2401] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:28:59.237101 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62597662-25b3-411c-8f36-5bbc4af8ec71 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:59.257716 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62597662-25b3-411c-8f36-5bbc4af8ec71 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:59.262957 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c42c44-a6c4-45bc-b8b6-19e240a01503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:59.263989 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c42c44-a6c4-45bc-b8b6-19e240a01503 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:59.264451 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c42c44-a6c4-45bc-b8b6-19e240a01503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:59.264738 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c42c44-a6c4-45bc-b8b6-19e240a01503 admin admin] PATH_INFO: `/v3/projects/08b0656631f04d6cacf20bc3e211ca4f/users/32a8f104bb874965ae146dfe800a6037/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:59.284804 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c42c44-a6c4-45bc-b8b6-19e240a01503 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08b0656631f04d6cacf20bc3e211ca4f, user_id=32a8f104bb874965ae146dfe800a6037, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:59.287741 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c42c44-a6c4-45bc-b8b6-19e240a01503 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:59.365711 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1183/2402] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:28:59 2022] PUT /identity/v3/projects/08b0656631f04d6cacf20bc3e211ca4f/users/32a8f104bb874965ae146dfe800a6037/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:28:59.377002 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c686b-04dc-407b-904a-bec91e48f0d0 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:59.377285 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c686b-04dc-407b-904a-bec91e48f0d0 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:59.377549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c686b-04dc-407b-904a-bec91e48f0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:59.387174 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:28:59.409403 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-618c686b-04dc-407b-904a-bec91e48f0d0 None None] MFA Rules not processed for user `32a8f104bb874965ae146dfe800a6037`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:28:59.410625 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-618c686b-04dc-407b-904a-bec91e48f0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:59.447144 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ecb98-0bdf-4e19-80c0-c2fa832fedd1 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:59.447665 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ecb98-0bdf-4e19-80c0-c2fa832fedd1 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:59.448100 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ecb98-0bdf-4e19-80c0-c2fa832fedd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:59.480584 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-e55ecb98-0bdf-4e19-80c0-c2fa832fedd1 None None] MFA Rules not processed for user `17e3b509a2de4159bf234594573f5e18`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:28:59.486493 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e55ecb98-0bdf-4e19-80c0-c2fa832fedd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:59.537592 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1220/2403] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:28:59 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:28:59.550549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c686b-04dc-407b-904a-bec91e48f0d0 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:59.552283 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-618c686b-04dc-407b-904a-bec91e48f0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:59.567744 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1184/2404] 10.222.0.26 () {58 vars in 1009 bytes} [Fri Aug 5 14:28:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:28:59.614458 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c686b-04dc-407b-904a-bec91e48f0d0 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:59.619213 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56656f08-46d9-4012-9fbd-160e10231d4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:59.620029 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56656f08-46d9-4012-9fbd-160e10231d4f service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:59.620231 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56656f08-46d9-4012-9fbd-160e10231d4f service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:59.620556 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56656f08-46d9-4012-9fbd-160e10231d4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:59.630529 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56656f08-46d9-4012-9fbd-160e10231d4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:59.631981 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56656f08-46d9-4012-9fbd-160e10231d4f service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:59.642504 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1221/2405] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:28:59 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 14:28:59.695799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ecb98-0bdf-4e19-80c0-c2fa832fedd1 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:28:59.697242 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e55ecb98-0bdf-4e19-80c0-c2fa832fedd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:28:59.787170 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ecb98-0bdf-4e19-80c0-c2fa832fedd1 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:28:59.792722 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c8ac22-7b68-495f-89c4-97b61f04916f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dddd7eea67f34f0cbb5f6a1cd85f1fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:28:59.793464 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8ac22-7b68-495f-89c4-97b61f04916f service swift] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:28:59.793815 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8ac22-7b68-495f-89c4-97b61f04916f service swift] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:28:59.794166 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8ac22-7b68-495f-89c4-97b61f04916f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:28:59.805231 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c8ac22-7b68-495f-89c4-97b61f04916f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:28:59.806955 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c8ac22-7b68-495f-89c4-97b61f04916f service swift] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:28:59.824253 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1185/2406] 10.222.0.26 () {60 vars in 1384 bytes} [Fri Aug 5 14:28:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 14:29:09.637749 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=56 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:29:09.813779 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=84,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:29:12.433610 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56656f08-46d9-4012-9fbd-160e10231d4f service cinder] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:29:12.499659 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56656f08-46d9-4012-9fbd-160e10231d4f service cinder] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:29:12.507990 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0aad3d-9734-4fb4-81c7-7bb4b778d2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:29:12.508677 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0aad3d-9734-4fb4-81c7-7bb4b778d2da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:29:12.509135 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0aad3d-9734-4fb4-81c7-7bb4b778d2da admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:29:12.509514 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0aad3d-9734-4fb4-81c7-7bb4b778d2da admin admin] PATH_INFO: `/v3/users/32a8f104bb874965ae146dfe800a6037` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:29:12.510806 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0aad3d-9734-4fb4-81c7-7bb4b778d2da admin admin] RBAC: Authorizing `identity:delete_user(user_id=32a8f104bb874965ae146dfe800a6037)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:29:12.512759 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0aad3d-9734-4fb4-81c7-7bb4b778d2da admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:29:12.744910 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-bb0aad3d-9734-4fb4-81c7-7bb4b778d2da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32a8f104bb874965ae146dfe800a6037'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:29:12.751848 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-bb0aad3d-9734-4fb4-81c7-7bb4b778d2da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32a8f104bb874965ae146dfe800a6037'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:29:12.752549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-bb0aad3d-9734-4fb4-81c7-7bb4b778d2da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32a8f104bb874965ae146dfe800a6037'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:29:12.753190 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-bb0aad3d-9734-4fb4-81c7-7bb4b778d2da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32a8f104bb874965ae146dfe800a6037'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:29:12.766916 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-bb0aad3d-9734-4fb4-81c7-7bb4b778d2da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32a8f104bb874965ae146dfe800a6037'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:29:12.844271 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1222/2407] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:29:12 2022] DELETE /identity/v3/users/32a8f104bb874965ae146dfe800a6037 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:29:12.848857 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f96c8d-30be-430f-962a-2a8fb88f2256 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:29:12.850575 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f96c8d-30be-430f-962a-2a8fb88f2256 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:29:12.850575 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f96c8d-30be-430f-962a-2a8fb88f2256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:29:12.869764 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-e9f96c8d-30be-430f-962a-2a8fb88f2256 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:29:12.871021 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e9f96c8d-30be-430f-962a-2a8fb88f2256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:29:12.928912 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1186/2408] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:29:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:29:12.938677 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0aad3d-9734-4fb4-81c7-7bb4b778d2da admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:29:12.940150 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-bb0aad3d-9734-4fb4-81c7-7bb4b778d2da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:29:12.998399 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0aad3d-9734-4fb4-81c7-7bb4b778d2da admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:29:13.001964 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d4c5f-5390-4949-84eb-6b664919fe29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:29:13.002599 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d4c5f-5390-4949-84eb-6b664919fe29 service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:29:13.002881 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d4c5f-5390-4949-84eb-6b664919fe29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:29:13.003156 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d4c5f-5390-4949-84eb-6b664919fe29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:29:13.012283 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d4c5f-5390-4949-84eb-6b664919fe29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:29:13.013730 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d4c5f-5390-4949-84eb-6b664919fe29 service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:29:13.024029 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1223/2409] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:29:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:29:13.682868 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f96c8d-30be-430f-962a-2a8fb88f2256 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:29:13.682868 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e9f96c8d-30be-430f-962a-2a8fb88f2256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:29:13.698832 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f96c8d-30be-430f-962a-2a8fb88f2256 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:29:13.704245 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffff2fc-bdee-4e6b-ab1d-e98feb8012e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:29:13.705339 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffff2fc-bdee-4e6b-ab1d-e98feb8012e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:29:13.706023 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffff2fc-bdee-4e6b-ab1d-e98feb8012e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:29:13.706446 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffff2fc-bdee-4e6b-ab1d-e98feb8012e1 admin admin] PATH_INFO: `/v3/projects/08b0656631f04d6cacf20bc3e211ca4f` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:29:13.708155 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffff2fc-bdee-4e6b-ab1d-e98feb8012e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08b0656631f04d6cacf20bc3e211ca4f)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:29:13.711047 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffff2fc-bdee-4e6b-ab1d-e98feb8012e1 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:29:13.851934 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7ffff2fc-bdee-4e6b-ab1d-e98feb8012e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08b0656631f04d6cacf20bc3e211ca4f', 'request_id': 'req-7ffff2fc-bdee-4e6b-ab1d-e98feb8012e1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:29:13.858819 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7ffff2fc-bdee-4e6b-ab1d-e98feb8012e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08b0656631f04d6cacf20bc3e211ca4f', 'request_id': 'req-7ffff2fc-bdee-4e6b-ab1d-e98feb8012e1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:29:13.918654 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7ffff2fc-bdee-4e6b-ab1d-e98feb8012e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08b0656631f04d6cacf20bc3e211ca4f', 'request_id': 'req-7ffff2fc-bdee-4e6b-ab1d-e98feb8012e1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:29:13.930657 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7ffff2fc-bdee-4e6b-ab1d-e98feb8012e1 admin admin] The token cache is being invalidate because project 08b0656631f04d6cacf20bc3e211ca4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:29:13.931020 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7ffff2fc-bdee-4e6b-ab1d-e98feb8012e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:29:13.931754 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7ffff2fc-bdee-4e6b-ab1d-e98feb8012e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:29:13.933282 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1187/2410] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:29:13 2022] DELETE /identity/v3/projects/08b0656631f04d6cacf20bc3e211ca4f => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:29:15.313677 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d4c5f-5390-4949-84eb-6b664919fe29 service neutron] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:29:15.322856 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d4c5f-5390-4949-84eb-6b664919fe29 service neutron] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:29:15.342004 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:29:15.371147 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d552bbcb-2b10-4f60-a55d-87a7a0891e50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:29:15.372012 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552bbcb-2b10-4f60-a55d-87a7a0891e50 service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:29:15.372649 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552bbcb-2b10-4f60-a55d-87a7a0891e50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:29:15.372985 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552bbcb-2b10-4f60-a55d-87a7a0891e50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:29:15.374779 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-d552bbcb-2b10-4f60-a55d-87a7a0891e50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:29:15.436384 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d552bbcb-2b10-4f60-a55d-87a7a0891e50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:29:15.437833 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d552bbcb-2b10-4f60-a55d-87a7a0891e50 service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:29:15.447148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1224/2411] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:29:15 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:29:19.875110 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e0c551-5764-4d94-a687-b8ff92918942 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:29:19.875661 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e0c551-5764-4d94-a687-b8ff92918942 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:29:19.875661 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e0c551-5764-4d94-a687-b8ff92918942 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:29:19.877628 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1188/2412] 10.222.0.26 () {54 vars in 921 bytes} [Fri Aug 5 14:29:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:29:19.884778 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f02aa-0548-4bfa-8e5a-084521fdab46 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:29:19.891871 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f02aa-0548-4bfa-8e5a-084521fdab46 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:29:19.891871 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f02aa-0548-4bfa-8e5a-084521fdab46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:29:19.932594 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-438f02aa-0548-4bfa-8e5a-084521fdab46 None None] MFA Rules not processed for user `5ec4940c99fc4fa8a5bc7cdabc4558fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:29:19.934196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-438f02aa-0548-4bfa-8e5a-084521fdab46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:29:20.009740 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1225/2413] 10.222.0.26 () {58 vars in 1018 bytes} [Fri Aug 5 14:29:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:29:20.023829 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e0c551-5764-4d94-a687-b8ff92918942 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:29:20.025833 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-b7e0c551-5764-4d94-a687-b8ff92918942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:29:20.093456 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e0c551-5764-4d94-a687-b8ff92918942 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:29:20.098310 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283ab74-13bb-432f-af7a-8925c2539d5c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1ce2a2b4eb405b999d3fb756795176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:29:20.098930 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283ab74-13bb-432f-af7a-8925c2539d5c service glance] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:29:20.099211 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283ab74-13bb-432f-af7a-8925c2539d5c service glance] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:29:20.099483 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283ab74-13bb-432f-af7a-8925c2539d5c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:29:20.117153 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283ab74-13bb-432f-af7a-8925c2539d5c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:29:20.118671 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283ab74-13bb-432f-af7a-8925c2539d5c service glance] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:29:20.132198 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1189/2414] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:29:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:29:20.346704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b24460b-d70e-4906-bd51-0164c9e9b6f6 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:29:20.347054 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b24460b-d70e-4906-bd51-0164c9e9b6f6 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:29:20.347747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b24460b-d70e-4906-bd51-0164c9e9b6f6 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:29:20.348660 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1226/2415] 10.222.0.26 () {54 vars in 921 bytes} [Fri Aug 5 14:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:29:20.359795 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec6532-6344-4bc0-b294-b9d78129c562 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:29:20.360746 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec6532-6344-4bc0-b294-b9d78129c562 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:29:20.361676 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec6532-6344-4bc0-b294-b9d78129c562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:29:20.384878 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-ddec6532-6344-4bc0-b294-b9d78129c562 None None] MFA Rules not processed for user `695e37560f8f495e987d1bd17acc2680`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:29:20.386092 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ddec6532-6344-4bc0-b294-b9d78129c562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:29:20.393193 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1190/2416] 10.222.0.26 () {58 vars in 1018 bytes} [Fri Aug 5 14:29:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:29:20.406137 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b24460b-d70e-4906-bd51-0164c9e9b6f6 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:29:20.407866 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-6b24460b-d70e-4906-bd51-0164c9e9b6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:29:20.422565 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b24460b-d70e-4906-bd51-0164c9e9b6f6 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:29:20.426254 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc5bd5-155b-43b6-afa3-0f32a0c1e174 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:29:20.426891 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc5bd5-155b-43b6-afa3-0f32a0c1e174 service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:29:20.427225 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc5bd5-155b-43b6-afa3-0f32a0c1e174 service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:29:20.427507 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc5bd5-155b-43b6-afa3-0f32a0c1e174 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:29:20.436256 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dc5bd5-155b-43b6-afa3-0f32a0c1e174 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:29:20.438551 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dc5bd5-155b-43b6-afa3-0f32a0c1e174 service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:29:20.447932 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1227/2417] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:29:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:29:20.541491 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2cc13-0a43-4ddd-b813-890f9fe29b07 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:29:20.541771 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2cc13-0a43-4ddd-b813-890f9fe29b07 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:29:20.549740 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2cc13-0a43-4ddd-b813-890f9fe29b07 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:29:20.549740 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1191/2418] 10.222.0.26 () {54 vars in 921 bytes} [Fri Aug 5 14:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:29:20.549953 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc9c66-1bab-49a2-a300-8a50c532c1e2 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:29:20.549953 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc9c66-1bab-49a2-a300-8a50c532c1e2 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:29:20.550128 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc9c66-1bab-49a2-a300-8a50c532c1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:29:20.570159 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-4bfc9c66-1bab-49a2-a300-8a50c532c1e2 None None] MFA Rules not processed for user `5ec4940c99fc4fa8a5bc7cdabc4558fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:29:20.571354 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-4bfc9c66-1bab-49a2-a300-8a50c532c1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:29:20.582237 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1228/2419] 10.222.0.26 () {58 vars in 1018 bytes} [Fri Aug 5 14:29:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:29:20.593728 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b2cc13-0a43-4ddd-b813-890f9fe29b07 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:29:20.604836 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b2cc13-0a43-4ddd-b813-890f9fe29b07 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:29:20.608465 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17266d-df75-4682-b0a1-922ba88c0d66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:29:20.609063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17266d-df75-4682-b0a1-922ba88c0d66 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:29:20.609333 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17266d-df75-4682-b0a1-922ba88c0d66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:29:20.609596 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17266d-df75-4682-b0a1-922ba88c0d66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:29:20.619564 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17266d-df75-4682-b0a1-922ba88c0d66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:29:20.621105 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17266d-df75-4682-b0a1-922ba88c0d66 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:29:20.632188 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1192/2420] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:29:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:29:24.972766 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ee7b3-907e-467d-806e-de88458638ff None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:29:24.973342 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ee7b3-907e-467d-806e-de88458638ff None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:29:24.973342 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ee7b3-907e-467d-806e-de88458638ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:29:24.993506 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-de7ee7b3-907e-467d-806e-de88458638ff None None] MFA Rules not processed for user `17e3b509a2de4159bf234594573f5e18`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:29:24.994713 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-de7ee7b3-907e-467d-806e-de88458638ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:29:25.034171 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:29:25.048682 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1229/2421] 10.222.0.26 () {58 vars in 1009 bytes} [Fri Aug 5 14:29:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:29:25.061173 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17266d-df75-4682-b0a1-922ba88c0d66 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:29:25.062567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-4a17266d-df75-4682-b0a1-922ba88c0d66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:29:25.072624 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:29:25.143033 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17266d-df75-4682-b0a1-922ba88c0d66 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:29:25.148074 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c6863-6478-4848-82a7-8cfdc12f4393 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dddd7eea67f34f0cbb5f6a1cd85f1fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:29:25.148712 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c6863-6478-4848-82a7-8cfdc12f4393 service swift] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:29:25.148986 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c6863-6478-4848-82a7-8cfdc12f4393 service swift] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:29:25.149253 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c6863-6478-4848-82a7-8cfdc12f4393 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:29:25.157820 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84c6863-6478-4848-82a7-8cfdc12f4393 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:29:25.159380 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84c6863-6478-4848-82a7-8cfdc12f4393 service swift] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:29:25.168030 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1193/2422] 10.222.0.26 () {60 vars in 1384 bytes} [Fri Aug 5 14:29:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 14:29:35.061302 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=5 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:29:35.165089 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=29 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:30:07.689678 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e5cf0-d6cf-45f8-8583-cdd5053cf886 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:30:07.690188 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e5cf0-d6cf-45f8-8583-cdd5053cf886 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:30:07.690288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e5cf0-d6cf-45f8-8583-cdd5053cf886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:30:07.709748 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-e39e5cf0-d6cf-45f8-8583-cdd5053cf886 None None] MFA Rules not processed for user `17e3b509a2de4159bf234594573f5e18`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:30:07.710940 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-e39e5cf0-d6cf-45f8-8583-cdd5053cf886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:30:07.719021 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1230/2423] 10.222.0.26 () {58 vars in 967 bytes} [Fri Aug 5 14:30:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:30:07.741447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c6863-6478-4848-82a7-8cfdc12f4393 service swift] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:30:07.752180 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c6863-6478-4848-82a7-8cfdc12f4393 service swift] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:30:07.757386 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae8d2a-482f-4ad0-af81-5bbf38c15be1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dddd7eea67f34f0cbb5f6a1cd85f1fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:30:07.758000 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae8d2a-482f-4ad0-af81-5bbf38c15be1 service swift] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:30:07.758276 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae8d2a-482f-4ad0-af81-5bbf38c15be1 service swift] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:30:07.758546 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae8d2a-482f-4ad0-af81-5bbf38c15be1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:30:07.773063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ae8d2a-482f-4ad0-af81-5bbf38c15be1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:30:07.774657 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ae8d2a-482f-4ad0-af81-5bbf38c15be1 service swift] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:30:07.784083 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1194/2424] 10.222.0.26 () {60 vars in 1384 bytes} [Fri Aug 5 14:30:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 14:30:10.985779 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e5cf0-d6cf-45f8-8583-cdd5053cf886 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:30:10.995390 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e5cf0-d6cf-45f8-8583-cdd5053cf886 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:30:10.999044 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511fce70-714c-4248-9249-b4dedc31e64e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:30:10.999655 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511fce70-714c-4248-9249-b4dedc31e64e service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:30:10.999929 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511fce70-714c-4248-9249-b4dedc31e64e service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:30:11.000196 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511fce70-714c-4248-9249-b4dedc31e64e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:30:11.003215 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-511fce70-714c-4248-9249-b4dedc31e64e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:30:11.018092 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511fce70-714c-4248-9249-b4dedc31e64e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:30:11.019493 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511fce70-714c-4248-9249-b4dedc31e64e service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:30:11.031271 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1231/2425] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:30:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:30:15.729623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae8d2a-482f-4ad0-af81-5bbf38c15be1 service swift] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:30:15.738758 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae8d2a-482f-4ad0-af81-5bbf38c15be1 service swift] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:30:15.742322 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74b4977-0b39-4793-adea-e7b1a66a27ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:30:15.742914 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b4977-0b39-4793-adea-e7b1a66a27ce service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:30:15.743187 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b4977-0b39-4793-adea-e7b1a66a27ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:30:15.743467 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b4977-0b39-4793-adea-e7b1a66a27ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:30:15.745384 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c74b4977-0b39-4793-adea-e7b1a66a27ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:30:15.802767 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74b4977-0b39-4793-adea-e7b1a66a27ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:30:15.804180 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74b4977-0b39-4793-adea-e7b1a66a27ce service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:30:15.813944 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1195/2426] 10.222.0.26 () {60 vars in 1396 bytes} [Fri Aug 5 14:30:15 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3268 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:30:16.992582 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511fce70-714c-4248-9249-b4dedc31e64e service neutron] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:30:17.001421 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511fce70-714c-4248-9249-b4dedc31e64e service neutron] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:30:17.005158 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc3d9af-9025-4316-959f-fd595758921d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:30:17.005759 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3d9af-9025-4316-959f-fd595758921d service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:30:17.006037 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3d9af-9025-4316-959f-fd595758921d service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:30:17.006310 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3d9af-9025-4316-959f-fd595758921d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:30:17.008036 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-4bc3d9af-9025-4316-959f-fd595758921d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:30:17.022744 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc3d9af-9025-4316-959f-fd595758921d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:30:17.024195 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc3d9af-9025-4316-959f-fd595758921d service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:30:17.036114 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1232/2427] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:30:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:30:25.808939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=34 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:30:27.031698 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=15 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:30:34.747408 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74b4977-0b39-4793-adea-e7b1a66a27ce service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:30:34.750056 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c74b4977-0b39-4793-adea-e7b1a66a27ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:30:34.765181 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74b4977-0b39-4793-adea-e7b1a66a27ce service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:30:34.771609 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70880036-3e91-40a3-b175-6c85d2aece6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:30:34.772368 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70880036-3e91-40a3-b175-6c85d2aece6c service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:30:34.772771 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70880036-3e91-40a3-b175-6c85d2aece6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:30:34.773146 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70880036-3e91-40a3-b175-6c85d2aece6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:30:34.775071 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-70880036-3e91-40a3-b175-6c85d2aece6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:30:34.786511 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70880036-3e91-40a3-b175-6c85d2aece6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:30:34.788037 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70880036-3e91-40a3-b175-6c85d2aece6c service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:30:34.796132 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1196/2428] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:30:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:30:40.139104 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111171b-6368-4de7-a252-2f77b826d852 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:30:40.139603 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111171b-6368-4de7-a252-2f77b826d852 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:30:40.139853 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111171b-6368-4de7-a252-2f77b826d852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:30:40.159963 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-4111171b-6368-4de7-a252-2f77b826d852 None None] MFA Rules not processed for user `cee6d9ed0c344584b8bf2fbe3c4178bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:30:40.161178 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-4111171b-6368-4de7-a252-2f77b826d852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:30:40.218955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1233/2429] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:30:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:30:40.713396 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70880036-3e91-40a3-b175-6c85d2aece6c service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:30:40.726987 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70880036-3e91-40a3-b175-6c85d2aece6c service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:30:40.732896 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0386314-2d97-4ade-8720-bf153e725aea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:30:40.735872 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0386314-2d97-4ade-8720-bf153e725aea service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:30:40.736155 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0386314-2d97-4ade-8720-bf153e725aea service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:30:40.736444 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0386314-2d97-4ade-8720-bf153e725aea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:30:40.738391 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f0386314-2d97-4ade-8720-bf153e725aea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:30:40.751870 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0386314-2d97-4ade-8720-bf153e725aea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:30:40.753344 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0386314-2d97-4ade-8720-bf153e725aea service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:30:40.763847 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1197/2430] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:30:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:30:41.229030 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4111171b-6368-4de7-a252-2f77b826d852 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:30:41.231444 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-4111171b-6368-4de7-a252-2f77b826d852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:30:41.319707 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4111171b-6368-4de7-a252-2f77b826d852 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:30:41.324812 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d47389e-3775-4cbb-929c-6a0dd0762631 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff761f6f81a46098f94bd2ec2be9c1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:30:41.325426 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47389e-3775-4cbb-929c-6a0dd0762631 service placement] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:30:41.325725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47389e-3775-4cbb-929c-6a0dd0762631 service placement] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:30:41.325995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47389e-3775-4cbb-929c-6a0dd0762631 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:30:41.327729 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-4d47389e-3775-4cbb-929c-6a0dd0762631 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:30:41.339741 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d47389e-3775-4cbb-929c-6a0dd0762631 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:30:41.341619 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d47389e-3775-4cbb-929c-6a0dd0762631 service placement] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:30:41.352508 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1234/2431] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:30:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:30:42.049088 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d6433-ae1f-47e6-94f0-fe6a674538e3 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:30:42.049619 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d6433-ae1f-47e6-94f0-fe6a674538e3 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:30:42.049719 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d6433-ae1f-47e6-94f0-fe6a674538e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:30:42.072477 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-3c0d6433-ae1f-47e6-94f0-fe6a674538e3 None None] MFA Rules not processed for user `b9d62fe4e6de4308a3a51be3e900625f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:30:42.073839 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-3c0d6433-ae1f-47e6-94f0-fe6a674538e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:30:42.081769 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1198/2432] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:30:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:30:46.990358 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d47389e-3775-4cbb-929c-6a0dd0762631 service placement] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:30:46.993129 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-4d47389e-3775-4cbb-929c-6a0dd0762631 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:30:47.015204 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d47389e-3775-4cbb-929c-6a0dd0762631 service placement] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:30:47.025107 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f0ef0-7d8a-482e-af07-682011d6dc90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:30:47.026044 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f0ef0-7d8a-482e-af07-682011d6dc90 service nova] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:30:47.026536 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f0ef0-7d8a-482e-af07-682011d6dc90 service nova] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:30:47.027541 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f0ef0-7d8a-482e-af07-682011d6dc90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:30:47.030169 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-8f9f0ef0-7d8a-482e-af07-682011d6dc90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:30:47.046261 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9f0ef0-7d8a-482e-af07-682011d6dc90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:30:47.049512 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9f0ef0-7d8a-482e-af07-682011d6dc90 service nova] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:30:47.062039 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1235/2433] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:30:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:30:48.557525 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d6433-ae1f-47e6-94f0-fe6a674538e3 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:30:48.571872 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d6433-ae1f-47e6-94f0-fe6a674538e3 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:30:48.577937 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ba8e2-9971-4793-8bf0-664a990577db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:30:48.578558 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ba8e2-9971-4793-8bf0-664a990577db service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:30:48.578839 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ba8e2-9971-4793-8bf0-664a990577db service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:30:48.579114 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ba8e2-9971-4793-8bf0-664a990577db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:30:48.581507 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-189ba8e2-9971-4793-8bf0-664a990577db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:30:48.595182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189ba8e2-9971-4793-8bf0-664a990577db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:30:48.597188 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189ba8e2-9971-4793-8bf0-664a990577db service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:30:48.607867 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1199/2434] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:30:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:30:55.242538 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f0ef0-7d8a-482e-af07-682011d6dc90 service nova] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:30:55.255813 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f0ef0-7d8a-482e-af07-682011d6dc90 service nova] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:30:55.261549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ac495-90cc-4b75-88a9-50f3473e3cf1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:30:55.262172 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ac495-90cc-4b75-88a9-50f3473e3cf1 service nova] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:30:55.262589 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ac495-90cc-4b75-88a9-50f3473e3cf1 service nova] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:30:55.262871 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ac495-90cc-4b75-88a9-50f3473e3cf1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:30:55.264684 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-4b9ac495-90cc-4b75-88a9-50f3473e3cf1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:30:55.277583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9ac495-90cc-4b75-88a9-50f3473e3cf1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:30:55.279154 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9ac495-90cc-4b75-88a9-50f3473e3cf1 service nova] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:30:55.290670 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1236/2435] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:30:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:30:58.603228 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=21 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:31:03.744324 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ba8e2-9971-4793-8bf0-664a990577db service nova] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:31:03.755439 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ba8e2-9971-4793-8bf0-664a990577db service nova] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:31:03.761684 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41135933-7bc8-4028-8287-7cea547f4282 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:31:03.762285 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41135933-7bc8-4028-8287-7cea547f4282 service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:31:03.762565 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41135933-7bc8-4028-8287-7cea547f4282 service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:31:03.762839 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41135933-7bc8-4028-8287-7cea547f4282 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:31:03.764687 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-41135933-7bc8-4028-8287-7cea547f4282 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:31:03.821958 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41135933-7bc8-4028-8287-7cea547f4282 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:31:03.823455 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41135933-7bc8-4028-8287-7cea547f4282 service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:31:03.832472 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1200/2436] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:31:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:31:05.286726 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=69 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:31:13.828257 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=28 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:31:39.617526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ac495-90cc-4b75-88a9-50f3473e3cf1 service nova] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:31:39.629030 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ac495-90cc-4b75-88a9-50f3473e3cf1 service nova] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:31:39.634807 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde83ab4-7dfd-4212-982d-9ef1571c4561 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:31:39.640646 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde83ab4-7dfd-4212-982d-9ef1571c4561 service nova] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:31:39.640646 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde83ab4-7dfd-4212-982d-9ef1571c4561 service nova] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:31:39.640646 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde83ab4-7dfd-4212-982d-9ef1571c4561 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:31:39.640646 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-fde83ab4-7dfd-4212-982d-9ef1571c4561 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:31:39.697182 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde83ab4-7dfd-4212-982d-9ef1571c4561 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:31:39.699443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde83ab4-7dfd-4212-982d-9ef1571c4561 service nova] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:31:39.709014 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1237/2437] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:31:39 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:31:49.704428 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=28 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:31:55.258103 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b84b8c-787e-4e70-b654-85aee72d70ee None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:31:55.258618 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b84b8c-787e-4e70-b654-85aee72d70ee None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:31:55.258717 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b84b8c-787e-4e70-b654-85aee72d70ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:31:55.278179 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-e1b84b8c-787e-4e70-b654-85aee72d70ee None None] MFA Rules not processed for user `084b18670c2a4c1396d5912d130b2bac`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:31:55.279386 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e1b84b8c-787e-4e70-b654-85aee72d70ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:31:55.286314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1201/2438] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:31:55 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:31:55.590735 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde83ab4-7dfd-4212-982d-9ef1571c4561 service nova] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:31:55.600748 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde83ab4-7dfd-4212-982d-9ef1571c4561 service nova] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:31:55.604314 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53faca8f-1d61-454e-a500-4039643778bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:31:55.604954 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53faca8f-1d61-454e-a500-4039643778bf service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:31:55.605232 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53faca8f-1d61-454e-a500-4039643778bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:31:55.605502 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53faca8f-1d61-454e-a500-4039643778bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:31:55.607232 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-53faca8f-1d61-454e-a500-4039643778bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:31:55.616694 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53faca8f-1d61-454e-a500-4039643778bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:31:55.618109 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53faca8f-1d61-454e-a500-4039643778bf service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:31:55.625986 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1238/2439] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:31:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:32:02.988799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b84b8c-787e-4e70-b654-85aee72d70ee None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:32:02.990294 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e1b84b8c-787e-4e70-b654-85aee72d70ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:32:03.006584 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b84b8c-787e-4e70-b654-85aee72d70ee None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:32:03.014073 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dcbe10-a461-4db8-8ba6-415e8ccd19b0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff761f6f81a46098f94bd2ec2be9c1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:32:03.014073 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dcbe10-a461-4db8-8ba6-415e8ccd19b0 service placement] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:03.014073 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dcbe10-a461-4db8-8ba6-415e8ccd19b0 service placement] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:03.014073 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dcbe10-a461-4db8-8ba6-415e8ccd19b0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:03.014513 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-86dcbe10-a461-4db8-8ba6-415e8ccd19b0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:32:03.026141 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dcbe10-a461-4db8-8ba6-415e8ccd19b0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:32:03.027730 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dcbe10-a461-4db8-8ba6-415e8ccd19b0 service placement] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:32:03.038677 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1202/2440] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:32:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:32:03.250422 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53faca8f-1d61-454e-a500-4039643778bf service neutron] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:32:03.251798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-53faca8f-1d61-454e-a500-4039643778bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:32:03.321120 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53faca8f-1d61-454e-a500-4039643778bf service neutron] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:32:03.326503 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a786c68-b1ba-45cb-9457-39d4028af047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:32:03.327162 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a786c68-b1ba-45cb-9457-39d4028af047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:03.327442 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a786c68-b1ba-45cb-9457-39d4028af047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:03.327719 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a786c68-b1ba-45cb-9457-39d4028af047 admin admin] PATH_INFO: `/v3/users/cee6d9ed0c344584b8bf2fbe3c4178bf` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:03.329003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a786c68-b1ba-45cb-9457-39d4028af047 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cee6d9ed0c344584b8bf2fbe3c4178bf)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:32:03.330983 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a786c68-b1ba-45cb-9457-39d4028af047 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:32:03.544873 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-7a786c68-b1ba-45cb-9457-39d4028af047 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cee6d9ed0c344584b8bf2fbe3c4178bf'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:03.552347 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-7a786c68-b1ba-45cb-9457-39d4028af047 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cee6d9ed0c344584b8bf2fbe3c4178bf'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:03.553092 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-7a786c68-b1ba-45cb-9457-39d4028af047 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cee6d9ed0c344584b8bf2fbe3c4178bf'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:03.553706 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-7a786c68-b1ba-45cb-9457-39d4028af047 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cee6d9ed0c344584b8bf2fbe3c4178bf'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:03.565581 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-7a786c68-b1ba-45cb-9457-39d4028af047 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cee6d9ed0c344584b8bf2fbe3c4178bf'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:03.637487 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1239/2441] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:32:03 2022] DELETE /identity/v3/users/cee6d9ed0c344584b8bf2fbe3c4178bf => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:32:03.988944 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dcbe10-a461-4db8-8ba6-415e8ccd19b0 service placement] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:32:04.000705 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dcbe10-a461-4db8-8ba6-415e8ccd19b0 service placement] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:32:04.072277 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0400dc-7e08-46c7-93d5-0893bebf4800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:32:04.072971 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0400dc-7e08-46c7-93d5-0893bebf4800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:04.073283 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0400dc-7e08-46c7-93d5-0893bebf4800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:04.073552 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0400dc-7e08-46c7-93d5-0893bebf4800 admin admin] PATH_INFO: `/v3/projects/5e50310a4bcd4dd5acde809956702d0b` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:04.074700 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0400dc-7e08-46c7-93d5-0893bebf4800 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e50310a4bcd4dd5acde809956702d0b)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:32:04.077996 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0400dc-7e08-46c7-93d5-0893bebf4800 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:32:04.198620 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-db0400dc-7e08-46c7-93d5-0893bebf4800 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e50310a4bcd4dd5acde809956702d0b', 'request_id': 'req-db0400dc-7e08-46c7-93d5-0893bebf4800'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:04.204861 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-db0400dc-7e08-46c7-93d5-0893bebf4800 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e50310a4bcd4dd5acde809956702d0b', 'request_id': 'req-db0400dc-7e08-46c7-93d5-0893bebf4800'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:04.272190 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-db0400dc-7e08-46c7-93d5-0893bebf4800 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e50310a4bcd4dd5acde809956702d0b', 'request_id': 'req-db0400dc-7e08-46c7-93d5-0893bebf4800'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:04.285515 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-db0400dc-7e08-46c7-93d5-0893bebf4800 admin admin] The token cache is being invalidate because project 5e50310a4bcd4dd5acde809956702d0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:32:04.285894 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-db0400dc-7e08-46c7-93d5-0893bebf4800 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:04.286677 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-db0400dc-7e08-46c7-93d5-0893bebf4800 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:04.288405 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1203/2442] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:32:03 2022] DELETE /identity/v3/projects/5e50310a4bcd4dd5acde809956702d0b => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:32:13.633791 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=59,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:32:14.280210 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=60,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:32:28.782323 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a786c68-b1ba-45cb-9457-39d4028af047 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:32:28.783737 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-7a786c68-b1ba-45cb-9457-39d4028af047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:32:28.841346 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a786c68-b1ba-45cb-9457-39d4028af047 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:32:28.889161 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697354a-06c8-4001-bca8-43628da017c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:32:28.889758 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697354a-06c8-4001-bca8-43628da017c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:28.890039 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697354a-06c8-4001-bca8-43628da017c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:28.890314 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697354a-06c8-4001-bca8-43628da017c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:28.892086 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-6697354a-06c8-4001-bca8-43628da017c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:32:28.954827 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6697354a-06c8-4001-bca8-43628da017c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:32:28.956262 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6697354a-06c8-4001-bca8-43628da017c2 service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:32:28.966950 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1240/2443] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:32:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:32:33.638501 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0400dc-7e08-46c7-93d5-0893bebf4800 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:32:33.639933 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-db0400dc-7e08-46c7-93d5-0893bebf4800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:32:33.654804 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0400dc-7e08-46c7-93d5-0893bebf4800 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:32:33.659369 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e869e4b-e934-4d74-a736-00e781345308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:32:33.660073 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e869e4b-e934-4d74-a736-00e781345308 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:33.660946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e869e4b-e934-4d74-a736-00e781345308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:33.661441 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e869e4b-e934-4d74-a736-00e781345308 admin admin] PATH_INFO: `/v3/users/b9d62fe4e6de4308a3a51be3e900625f` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:33.662801 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e869e4b-e934-4d74-a736-00e781345308 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9d62fe4e6de4308a3a51be3e900625f)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:32:33.664727 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e869e4b-e934-4d74-a736-00e781345308 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:32:33.898463 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9e869e4b-e934-4d74-a736-00e781345308 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9d62fe4e6de4308a3a51be3e900625f'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:33.911156 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9e869e4b-e934-4d74-a736-00e781345308 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9d62fe4e6de4308a3a51be3e900625f'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:33.975954 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9e869e4b-e934-4d74-a736-00e781345308 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9d62fe4e6de4308a3a51be3e900625f'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:33.976869 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9e869e4b-e934-4d74-a736-00e781345308 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9d62fe4e6de4308a3a51be3e900625f'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:33.985476 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9e869e4b-e934-4d74-a736-00e781345308 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9d62fe4e6de4308a3a51be3e900625f'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:33.993233 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1204/2444] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:32:33 2022] DELETE /identity/v3/users/b9d62fe4e6de4308a3a51be3e900625f => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:32:34.316565 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697354a-06c8-4001-bca8-43628da017c2 service neutron] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:32:34.330488 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697354a-06c8-4001-bca8-43628da017c2 service neutron] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:32:34.423041 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d683baa7-6bb7-4de1-941a-9067d65e363f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:32:34.424048 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683baa7-6bb7-4de1-941a-9067d65e363f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:34.424556 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683baa7-6bb7-4de1-941a-9067d65e363f admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:34.427092 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683baa7-6bb7-4de1-941a-9067d65e363f admin admin] PATH_INFO: `/v3/projects/8725df4ddc2b4ec8accc5b0bd7b39a49` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:34.427720 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d683baa7-6bb7-4de1-941a-9067d65e363f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8725df4ddc2b4ec8accc5b0bd7b39a49)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:32:34.429934 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d683baa7-6bb7-4de1-941a-9067d65e363f admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:32:34.439963 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:32:34.557996 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-d683baa7-6bb7-4de1-941a-9067d65e363f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8725df4ddc2b4ec8accc5b0bd7b39a49', 'request_id': 'req-d683baa7-6bb7-4de1-941a-9067d65e363f'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:34.564723 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-d683baa7-6bb7-4de1-941a-9067d65e363f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8725df4ddc2b4ec8accc5b0bd7b39a49', 'request_id': 'req-d683baa7-6bb7-4de1-941a-9067d65e363f'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:34.571751 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-d683baa7-6bb7-4de1-941a-9067d65e363f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8725df4ddc2b4ec8accc5b0bd7b39a49', 'request_id': 'req-d683baa7-6bb7-4de1-941a-9067d65e363f'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:34.643379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-d683baa7-6bb7-4de1-941a-9067d65e363f admin admin] The token cache is being invalidate because project 8725df4ddc2b4ec8accc5b0bd7b39a49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:32:34.646316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-d683baa7-6bb7-4de1-941a-9067d65e363f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:34.646316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-d683baa7-6bb7-4de1-941a-9067d65e363f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:34.646316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1241/2445] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:32:34 2022] DELETE /identity/v3/projects/8725df4ddc2b4ec8accc5b0bd7b39a49 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:32:34.651289 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8383d16-cfee-4671-a0cf-008faafc6a22 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:34.651804 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8383d16-cfee-4671-a0cf-008faafc6a22 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:34.651899 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8383d16-cfee-4671-a0cf-008faafc6a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:34.673874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-e8383d16-cfee-4671-a0cf-008faafc6a22 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:32:34.675003 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e8383d16-cfee-4671-a0cf-008faafc6a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:32:34.752870 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1205/2446] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:32:34.762045 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3d1f4-ba61-4ba6-927e-9100f709df65 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:34.762412 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3d1f4-ba61-4ba6-927e-9100f709df65 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:34.762802 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3d1f4-ba61-4ba6-927e-9100f709df65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:34.783891 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-edd3d1f4-ba61-4ba6-927e-9100f709df65 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:32:34.785193 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-edd3d1f4-ba61-4ba6-927e-9100f709df65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:32:34.794050 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1242/2447] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:32:34.798383 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8383d16-cfee-4671-a0cf-008faafc6a22 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:32:34.809613 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8383d16-cfee-4671-a0cf-008faafc6a22 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:32:34.814352 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b5f88-c47a-4393-a09b-465fed4dcee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:32:34.815175 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b5f88-c47a-4393-a09b-465fed4dcee1 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:34.815530 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b5f88-c47a-4393-a09b-465fed4dcee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:34.815868 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b5f88-c47a-4393-a09b-465fed4dcee1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:34.816737 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b5f88-c47a-4393-a09b-465fed4dcee1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:32:34.817228 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b5f88-c47a-4393-a09b-465fed4dcee1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:32:34.818819 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b5f88-c47a-4393-a09b-465fed4dcee1 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:32:34.831678 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1206/2448] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:32:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:32:34.836772 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3d1f4-ba61-4ba6-927e-9100f709df65 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:32:34.846258 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3d1f4-ba61-4ba6-927e-9100f709df65 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:32:34.851742 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f280f70-734a-4945-b7db-c88e6a8e178d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:32:34.852352 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f280f70-734a-4945-b7db-c88e6a8e178d admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:34.852939 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f280f70-734a-4945-b7db-c88e6a8e178d admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:34.853291 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f280f70-734a-4945-b7db-c88e6a8e178d admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:34.853962 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f280f70-734a-4945-b7db-c88e6a8e178d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:32:34.855855 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f280f70-734a-4945-b7db-c88e6a8e178d admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:32:34.952274 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1243/2449] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:32:34 2022] POST /identity/v3/projects => generated 405 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:32:34.953128 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b5f88-c47a-4393-a09b-465fed4dcee1 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:32:34.965146 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b5f88-c47a-4393-a09b-465fed4dcee1 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:32:35.026496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890263b-eae8-4a1d-8b5e-db709650d5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:32:35.027122 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890263b-eae8-4a1d-8b5e-db709650d5f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:35.027399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890263b-eae8-4a1d-8b5e-db709650d5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:35.027670 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890263b-eae8-4a1d-8b5e-db709650d5f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:35.028349 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5890263b-eae8-4a1d-8b5e-db709650d5f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:32:35.030303 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5890263b-eae8-4a1d-8b5e-db709650d5f3 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:32:35.038837 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:32:35.114116 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1207/2450] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:32:34 2022] POST /identity/v3/users => generated 303 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:32:35.117687 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f280f70-734a-4945-b7db-c88e6a8e178d admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:32:35.128095 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f280f70-734a-4945-b7db-c88e6a8e178d admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:32:35.133440 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75143e5a-ecb7-4ccd-87c0-250736bc9495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:32:35.135424 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75143e5a-ecb7-4ccd-87c0-250736bc9495 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:35.135711 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75143e5a-ecb7-4ccd-87c0-250736bc9495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:35.135982 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75143e5a-ecb7-4ccd-87c0-250736bc9495 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:35.136709 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75143e5a-ecb7-4ccd-87c0-250736bc9495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:32:35.138302 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75143e5a-ecb7-4ccd-87c0-250736bc9495 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:32:35.148932 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1244/2451] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:32:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:32:35.163196 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890263b-eae8-4a1d-8b5e-db709650d5f3 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:32:35.173452 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890263b-eae8-4a1d-8b5e-db709650d5f3 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:32:35.178905 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a0286-9db7-48f7-87e1-4ef33d5a37a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:32:35.179549 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a0286-9db7-48f7-87e1-4ef33d5a37a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:35.180004 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a0286-9db7-48f7-87e1-4ef33d5a37a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:35.180237 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a0286-9db7-48f7-87e1-4ef33d5a37a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:35.181594 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5a0286-9db7-48f7-87e1-4ef33d5a37a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:32:35.182835 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5a0286-9db7-48f7-87e1-4ef33d5a37a6 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:32:35.197917 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1208/2452] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:32:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:32:35.199194 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75143e5a-ecb7-4ccd-87c0-250736bc9495 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:32:35.209871 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75143e5a-ecb7-4ccd-87c0-250736bc9495 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:32:35.214709 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d0385-7a38-4702-8756-ea2e10633c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:32:35.220067 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d0385-7a38-4702-8756-ea2e10633c05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:35.220067 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d0385-7a38-4702-8756-ea2e10633c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:35.220067 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d0385-7a38-4702-8756-ea2e10633c05 admin admin] PATH_INFO: `/v3/projects/72d7854583064df7823bc8d6c2a4aac9/users/b9e64debd9104a7a9678c8c0960f2f5b/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:35.231459 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137d0385-7a38-4702-8756-ea2e10633c05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72d7854583064df7823bc8d6c2a4aac9, user_id=b9e64debd9104a7a9678c8c0960f2f5b, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:32:35.234295 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137d0385-7a38-4702-8756-ea2e10633c05 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:32:35.300378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1245/2453] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:32:35 2022] PUT /identity/v3/projects/72d7854583064df7823bc8d6c2a4aac9/users/b9e64debd9104a7a9678c8c0960f2f5b/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:32:35.306689 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebdef05-1eef-46ef-aa7e-2963119211e2 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:35.307002 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebdef05-1eef-46ef-aa7e-2963119211e2 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:35.307276 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebdef05-1eef-46ef-aa7e-2963119211e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:35.326792 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-3ebdef05-1eef-46ef-aa7e-2963119211e2 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:32:35.328002 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-3ebdef05-1eef-46ef-aa7e-2963119211e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:32:35.386464 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1209/2454] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:32:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:32:35.396228 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d0385-7a38-4702-8756-ea2e10633c05 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:32:35.397965 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-137d0385-7a38-4702-8756-ea2e10633c05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:32:35.456063 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d0385-7a38-4702-8756-ea2e10633c05 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:32:35.460503 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311d64ac-05be-4573-8299-6aa37f72c41e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:32:35.461115 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d64ac-05be-4573-8299-6aa37f72c41e service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:35.461392 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d64ac-05be-4573-8299-6aa37f72c41e service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:35.461685 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d64ac-05be-4573-8299-6aa37f72c41e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:35.470869 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311d64ac-05be-4573-8299-6aa37f72c41e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:32:35.472283 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311d64ac-05be-4573-8299-6aa37f72c41e service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:32:35.481838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1246/2455] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:32:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 14:32:36.859954 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b3d22-b416-4e89-97ef-101e9659284c None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:36.860510 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b3d22-b416-4e89-97ef-101e9659284c None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:36.860780 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b3d22-b416-4e89-97ef-101e9659284c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:36.893553 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-115b3d22-b416-4e89-97ef-101e9659284c None None] MFA Rules not processed for user `b9e64debd9104a7a9678c8c0960f2f5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:32:36.896121 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-115b3d22-b416-4e89-97ef-101e9659284c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:32:36.988523 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1210/2456] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:32:36 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:32:36.997744 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311d64ac-05be-4573-8299-6aa37f72c41e service neutron] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:32:37.008968 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311d64ac-05be-4573-8299-6aa37f72c41e service neutron] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:32:37.013932 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c0b90d-ec43-40a6-94e6-bbaa03c6bcb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:32:37.014558 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0b90d-ec43-40a6-94e6-bbaa03c6bcb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:37.014867 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0b90d-ec43-40a6-94e6-bbaa03c6bcb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:37.015142 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0b90d-ec43-40a6-94e6-bbaa03c6bcb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:37.015833 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c0b90d-ec43-40a6-94e6-bbaa03c6bcb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:32:37.017882 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c0b90d-ec43-40a6-94e6-bbaa03c6bcb4 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:32:37.099519 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1247/2457] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:32:36 2022] POST /identity/v3/projects => generated 405 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:32:37.100734 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115b3d22-b416-4e89-97ef-101e9659284c None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:32:37.111823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115b3d22-b416-4e89-97ef-101e9659284c None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:32:37.168843 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bea0ee-0370-4b3a-acd0-deb7611f3fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:32:37.169527 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bea0ee-0370-4b3a-acd0-deb7611f3fe7 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:37.169804 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bea0ee-0370-4b3a-acd0-deb7611f3fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:37.170079 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bea0ee-0370-4b3a-acd0-deb7611f3fe7 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:37.170751 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bea0ee-0370-4b3a-acd0-deb7611f3fe7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:32:37.172884 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bea0ee-0370-4b3a-acd0-deb7611f3fe7 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:32:37.261957 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:32:37.268074 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1211/2458] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:32:37 2022] POST /identity/v3/users => generated 309 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:32:37.273971 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c0b90d-ec43-40a6-94e6-bbaa03c6bcb4 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:32:37.284741 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c0b90d-ec43-40a6-94e6-bbaa03c6bcb4 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:32:37.289621 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece70b4-54c9-447e-b994-90c331a06d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:32:37.290237 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece70b4-54c9-447e-b994-90c331a06d60 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:37.290516 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece70b4-54c9-447e-b994-90c331a06d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:37.291032 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece70b4-54c9-447e-b994-90c331a06d60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:37.291773 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece70b4-54c9-447e-b994-90c331a06d60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:32:37.293407 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece70b4-54c9-447e-b994-90c331a06d60 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:32:37.305503 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1248/2459] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:32:37.309737 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bea0ee-0370-4b3a-acd0-deb7611f3fe7 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:32:37.321823 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bea0ee-0370-4b3a-acd0-deb7611f3fe7 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:32:37.326702 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a0b3e5-1694-4501-a55e-0c852f3f1514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:32:37.327374 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0b3e5-1694-4501-a55e-0c852f3f1514 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:37.327670 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0b3e5-1694-4501-a55e-0c852f3f1514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:37.327924 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0b3e5-1694-4501-a55e-0c852f3f1514 admin admin] PATH_INFO: `/v3/projects/c25e108b532548328ba5e983f12d22ee/users/c5dc5ab7cd784c7ea2921f9d71c30345/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:37.343124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a0b3e5-1694-4501-a55e-0c852f3f1514 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c25e108b532548328ba5e983f12d22ee, user_id=c5dc5ab7cd784c7ea2921f9d71c30345, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:32:37.345829 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a0b3e5-1694-4501-a55e-0c852f3f1514 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:32:37.416169 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1212/2460] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:32:37 2022] PUT /identity/v3/projects/c25e108b532548328ba5e983f12d22ee/users/c5dc5ab7cd784c7ea2921f9d71c30345/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:32:37.420169 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece70b4-54c9-447e-b994-90c331a06d60 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:32:37.425208 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:32:37.431059 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece70b4-54c9-447e-b994-90c331a06d60 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:32:37.485976 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a60c81-8cbd-4be0-9ba4-5b8591c0a7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:32:37.486592 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a60c81-8cbd-4be0-9ba4-5b8591c0a7ee admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:37.486871 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a60c81-8cbd-4be0-9ba4-5b8591c0a7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:37.487143 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a60c81-8cbd-4be0-9ba4-5b8591c0a7ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:37.487837 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a60c81-8cbd-4be0-9ba4-5b8591c0a7ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:32:37.490680 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a60c81-8cbd-4be0-9ba4-5b8591c0a7ee admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:32:37.501980 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1249/2461] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:32:37.507267 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a0b3e5-1694-4501-a55e-0c852f3f1514 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:32:37.518842 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a0b3e5-1694-4501-a55e-0c852f3f1514 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:32:37.523697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29586c71-14ca-4f79-94ab-6daecc18a7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:32:37.524360 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29586c71-14ca-4f79-94ab-6daecc18a7d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:37.524692 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29586c71-14ca-4f79-94ab-6daecc18a7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:37.524964 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29586c71-14ca-4f79-94ab-6daecc18a7d3 admin admin] PATH_INFO: `/v3/projects/c25e108b532548328ba5e983f12d22ee/users/c5dc5ab7cd784c7ea2921f9d71c30345/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:37.527134 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29586c71-14ca-4f79-94ab-6daecc18a7d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c25e108b532548328ba5e983f12d22ee, user_id=c5dc5ab7cd784c7ea2921f9d71c30345, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:32:37.529642 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29586c71-14ca-4f79-94ab-6daecc18a7d3 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:32:37.540400 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1213/2462] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:32:37 2022] PUT /identity/v3/projects/c25e108b532548328ba5e983f12d22ee/users/c5dc5ab7cd784c7ea2921f9d71c30345/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:32:38.738130 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc1f85-6811-4311-97d7-ecffa8e41f5b None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:38.739311 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc1f85-6811-4311-97d7-ecffa8e41f5b None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:38.739311 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc1f85-6811-4311-97d7-ecffa8e41f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:38.771702 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-1bdc1f85-6811-4311-97d7-ecffa8e41f5b None None] MFA Rules not processed for user `c5dc5ab7cd784c7ea2921f9d71c30345`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:32:38.772968 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-1bdc1f85-6811-4311-97d7-ecffa8e41f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:32:38.862422 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1250/2463] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:32:38 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:32:39.172856 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00abdfb-582e-4673-8e83-6faed5482d8c None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:39.173322 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00abdfb-582e-4673-8e83-6faed5482d8c None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:39.173441 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00abdfb-582e-4673-8e83-6faed5482d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:39.194957 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-f00abdfb-582e-4673-8e83-6faed5482d8c None None] MFA Rules not processed for user `99dc94fff06f42c39f3750484f6d8e9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:32:39.196270 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f00abdfb-582e-4673-8e83-6faed5482d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:32:39.272586 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1214/2464] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:32:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:32:40.507956 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc1f85-6811-4311-97d7-ecffa8e41f5b None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:32:40.700892 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc1f85-6811-4311-97d7-ecffa8e41f5b None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:32:40.705449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33264c9a-8e68-4b15-b8a1-bfe603eaaeb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:32:40.706092 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33264c9a-8e68-4b15-b8a1-bfe603eaaeb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:40.706371 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33264c9a-8e68-4b15-b8a1-bfe603eaaeb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:40.706644 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33264c9a-8e68-4b15-b8a1-bfe603eaaeb7 admin admin] PATH_INFO: `/v3/users/b9e64debd9104a7a9678c8c0960f2f5b` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:40.708471 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33264c9a-8e68-4b15-b8a1-bfe603eaaeb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9e64debd9104a7a9678c8c0960f2f5b)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:32:40.710396 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33264c9a-8e68-4b15-b8a1-bfe603eaaeb7 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:32:40.816484 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:32:40.914578 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-33264c9a-8e68-4b15-b8a1-bfe603eaaeb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9e64debd9104a7a9678c8c0960f2f5b'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:40.921165 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-33264c9a-8e68-4b15-b8a1-bfe603eaaeb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9e64debd9104a7a9678c8c0960f2f5b'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:40.921822 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-33264c9a-8e68-4b15-b8a1-bfe603eaaeb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9e64debd9104a7a9678c8c0960f2f5b'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:40.922438 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-33264c9a-8e68-4b15-b8a1-bfe603eaaeb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9e64debd9104a7a9678c8c0960f2f5b'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:40.933839 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-33264c9a-8e68-4b15-b8a1-bfe603eaaeb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9e64debd9104a7a9678c8c0960f2f5b'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:41.015696 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1251/2465] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:32:40 2022] DELETE /identity/v3/users/b9e64debd9104a7a9678c8c0960f2f5b => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:32:41.259619 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00abdfb-582e-4673-8e83-6faed5482d8c None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:32:41.276403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00abdfb-582e-4673-8e83-6faed5482d8c None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:32:41.350584 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e87b1-fc5c-4206-8087-9b925b978871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:32:41.351296 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e87b1-fc5c-4206-8087-9b925b978871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:41.351650 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e87b1-fc5c-4206-8087-9b925b978871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:41.352320 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e87b1-fc5c-4206-8087-9b925b978871 admin admin] PATH_INFO: `/v3/projects/72d7854583064df7823bc8d6c2a4aac9` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:41.353287 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30e87b1-fc5c-4206-8087-9b925b978871 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72d7854583064df7823bc8d6c2a4aac9)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:32:41.355550 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30e87b1-fc5c-4206-8087-9b925b978871 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:32:41.466180 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-d30e87b1-fc5c-4206-8087-9b925b978871 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72d7854583064df7823bc8d6c2a4aac9', 'request_id': 'req-d30e87b1-fc5c-4206-8087-9b925b978871'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:41.473311 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-d30e87b1-fc5c-4206-8087-9b925b978871 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72d7854583064df7823bc8d6c2a4aac9', 'request_id': 'req-d30e87b1-fc5c-4206-8087-9b925b978871'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:41.545693 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-d30e87b1-fc5c-4206-8087-9b925b978871 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72d7854583064df7823bc8d6c2a4aac9', 'request_id': 'req-d30e87b1-fc5c-4206-8087-9b925b978871'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:41.552196 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:32:41.559199 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-d30e87b1-fc5c-4206-8087-9b925b978871 admin admin] The token cache is being invalidate because project 72d7854583064df7823bc8d6c2a4aac9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:32:41.560569 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-d30e87b1-fc5c-4206-8087-9b925b978871 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:41.560699 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-d30e87b1-fc5c-4206-8087-9b925b978871 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:41.562156 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1215/2466] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:32:41 2022] DELETE /identity/v3/projects/72d7854583064df7823bc8d6c2a4aac9 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:32:41.566120 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33264c9a-8e68-4b15-b8a1-bfe603eaaeb7 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:32:41.567613 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-33264c9a-8e68-4b15-b8a1-bfe603eaaeb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:32:41.580639 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33264c9a-8e68-4b15-b8a1-bfe603eaaeb7 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:32:41.636831 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35d2b7e-fd58-4e0b-80b0-898676d2738a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:32:41.637467 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d2b7e-fd58-4e0b-80b0-898676d2738a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:41.637744 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d2b7e-fd58-4e0b-80b0-898676d2738a admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:41.638018 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d2b7e-fd58-4e0b-80b0-898676d2738a admin admin] PATH_INFO: `/v3/users/c5dc5ab7cd784c7ea2921f9d71c30345` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:41.639180 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35d2b7e-fd58-4e0b-80b0-898676d2738a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5dc5ab7cd784c7ea2921f9d71c30345)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:32:41.641100 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35d2b7e-fd58-4e0b-80b0-898676d2738a admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:32:41.830145 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-f35d2b7e-fd58-4e0b-80b0-898676d2738a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5dc5ab7cd784c7ea2921f9d71c30345'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:41.838337 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-f35d2b7e-fd58-4e0b-80b0-898676d2738a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5dc5ab7cd784c7ea2921f9d71c30345'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:41.839099 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-f35d2b7e-fd58-4e0b-80b0-898676d2738a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5dc5ab7cd784c7ea2921f9d71c30345'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:41.839758 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-f35d2b7e-fd58-4e0b-80b0-898676d2738a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5dc5ab7cd784c7ea2921f9d71c30345'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:41.853229 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-f35d2b7e-fd58-4e0b-80b0-898676d2738a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5dc5ab7cd784c7ea2921f9d71c30345'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:41.906764 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1252/2467] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:32:41 2022] DELETE /identity/v3/users/c5dc5ab7cd784c7ea2921f9d71c30345 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:32:42.154065 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e87b1-fc5c-4206-8087-9b925b978871 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:32:42.155560 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d30e87b1-fc5c-4206-8087-9b925b978871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:32:42.174789 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e87b1-fc5c-4206-8087-9b925b978871 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:32:42.237547 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f487a-d6e1-4b62-9709-5245fbdbbcca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:32:42.238218 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f487a-d6e1-4b62-9709-5245fbdbbcca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:42.238755 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f487a-d6e1-4b62-9709-5245fbdbbcca admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:42.239232 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f487a-d6e1-4b62-9709-5245fbdbbcca admin admin] PATH_INFO: `/v3/projects/c25e108b532548328ba5e983f12d22ee` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:42.240534 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f487a-d6e1-4b62-9709-5245fbdbbcca admin admin] RBAC: Authorizing `identity:delete_project(project_id=c25e108b532548328ba5e983f12d22ee)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:32:42.243012 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f487a-d6e1-4b62-9709-5245fbdbbcca admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:32:42.364462 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-277f487a-d6e1-4b62-9709-5245fbdbbcca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c25e108b532548328ba5e983f12d22ee', 'request_id': 'req-277f487a-d6e1-4b62-9709-5245fbdbbcca'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:42.370745 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-277f487a-d6e1-4b62-9709-5245fbdbbcca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c25e108b532548328ba5e983f12d22ee', 'request_id': 'req-277f487a-d6e1-4b62-9709-5245fbdbbcca'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:42.444219 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-277f487a-d6e1-4b62-9709-5245fbdbbcca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c25e108b532548328ba5e983f12d22ee', 'request_id': 'req-277f487a-d6e1-4b62-9709-5245fbdbbcca'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:42.455395 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-277f487a-d6e1-4b62-9709-5245fbdbbcca admin admin] The token cache is being invalidate because project c25e108b532548328ba5e983f12d22ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:32:42.455674 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-277f487a-d6e1-4b62-9709-5245fbdbbcca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:42.456296 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-277f487a-d6e1-4b62-9709-5245fbdbbcca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:32:42.458501 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1216/2468] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:32:42 2022] DELETE /identity/v3/projects/c25e108b532548328ba5e983f12d22ee => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:32:42.463148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f7530-f153-478c-bec1-9dd8201d880e None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:42.463433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f7530-f153-478c-bec1-9dd8201d880e None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:42.463705 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f7530-f153-478c-bec1-9dd8201d880e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:42.485140 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-0f3f7530-f153-478c-bec1-9dd8201d880e None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:32:42.486380 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-0f3f7530-f153-478c-bec1-9dd8201d880e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:32:42.506133 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:32:42.561151 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1253/2469] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:32:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:32:42.569507 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd4834-88aa-47e6-8b57-360de1163f43 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:42.569791 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd4834-88aa-47e6-8b57-360de1163f43 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:42.570064 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd4834-88aa-47e6-8b57-360de1163f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:42.589635 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-2abd4834-88aa-47e6-8b57-360de1163f43 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:32:42.590842 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-2abd4834-88aa-47e6-8b57-360de1163f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:32:42.599190 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1217/2470] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:32:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:32:42.609116 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f7530-f153-478c-bec1-9dd8201d880e None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:32:42.619272 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f7530-f153-478c-bec1-9dd8201d880e None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:32:42.623982 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c855c-5658-49aa-a224-e63da0ff8379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:32:42.624577 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c855c-5658-49aa-a224-e63da0ff8379 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:42.624856 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c855c-5658-49aa-a224-e63da0ff8379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:42.625140 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c855c-5658-49aa-a224-e63da0ff8379 admin admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:42.625828 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994c855c-5658-49aa-a224-e63da0ff8379 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:32:42.626129 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994c855c-5658-49aa-a224-e63da0ff8379 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:32:42.627724 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994c855c-5658-49aa-a224-e63da0ff8379 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:32:42.641719 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1254/2471] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:32:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:32:42.646693 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd4834-88aa-47e6-8b57-360de1163f43 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:32:42.656145 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd4834-88aa-47e6-8b57-360de1163f43 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:32:42.662478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c734d2-0706-4d9d-8752-8e48b8330776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:32:42.663123 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c734d2-0706-4d9d-8752-8e48b8330776 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:42.663393 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c734d2-0706-4d9d-8752-8e48b8330776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:42.663660 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c734d2-0706-4d9d-8752-8e48b8330776 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:42.664423 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c734d2-0706-4d9d-8752-8e48b8330776 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:32:42.666335 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c734d2-0706-4d9d-8752-8e48b8330776 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:32:42.778270 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1218/2472] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:32:42 2022] POST /identity/v3/projects => generated 371 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:32:42.782487 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c855c-5658-49aa-a224-e63da0ff8379 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:32:42.793688 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c855c-5658-49aa-a224-e63da0ff8379 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:32:42.853922 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23489bfd-4e49-424b-83b8-77e02dc7df2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:32:42.854712 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23489bfd-4e49-424b-83b8-77e02dc7df2b admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:42.854924 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23489bfd-4e49-424b-83b8-77e02dc7df2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:42.855195 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23489bfd-4e49-424b-83b8-77e02dc7df2b admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:42.855859 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23489bfd-4e49-424b-83b8-77e02dc7df2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:32:42.857782 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23489bfd-4e49-424b-83b8-77e02dc7df2b admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:32:43.000509 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1255/2473] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:32:42 2022] POST /identity/v3/users => generated 286 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:32:43.005832 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c734d2-0706-4d9d-8752-8e48b8330776 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:32:43.021754 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c734d2-0706-4d9d-8752-8e48b8330776 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:32:43.024952 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71df66ca-3ec1-4f9e-a1ac-948f2782f157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:32:43.025725 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df66ca-3ec1-4f9e-a1ac-948f2782f157 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:43.026070 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df66ca-3ec1-4f9e-a1ac-948f2782f157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:43.026349 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df66ca-3ec1-4f9e-a1ac-948f2782f157 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:43.027122 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71df66ca-3ec1-4f9e-a1ac-948f2782f157 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:32:43.028821 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71df66ca-3ec1-4f9e-a1ac-948f2782f157 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:32:43.039921 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1219/2474] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:32:43.044610 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23489bfd-4e49-424b-83b8-77e02dc7df2b admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:32:43.057926 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23489bfd-4e49-424b-83b8-77e02dc7df2b admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:32:43.065158 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d48c2-e8d4-4de8-8f4a-6f2473739f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:32:43.071715 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d48c2-e8d4-4de8-8f4a-6f2473739f7d admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:43.071715 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d48c2-e8d4-4de8-8f4a-6f2473739f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:43.071715 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d48c2-e8d4-4de8-8f4a-6f2473739f7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:43.071715 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496d48c2-e8d4-4de8-8f4a-6f2473739f7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:32:43.071715 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496d48c2-e8d4-4de8-8f4a-6f2473739f7d admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:32:43.091956 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1256/2475] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:32:43.092785 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71df66ca-3ec1-4f9e-a1ac-948f2782f157 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:32:43.103850 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71df66ca-3ec1-4f9e-a1ac-948f2782f157 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:32:43.115417 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b20abfb-9d2f-4212-ad84-7eafa0402c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:32:43.117025 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20abfb-9d2f-4212-ad84-7eafa0402c84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:43.117025 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20abfb-9d2f-4212-ad84-7eafa0402c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:43.117025 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20abfb-9d2f-4212-ad84-7eafa0402c84 admin admin] PATH_INFO: `/v3/projects/0335ef053e5845cd8ead785671a1b0f0/users/4e7cbe1bbe5c46e3b224d6bf67d9367a/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:43.135494 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b20abfb-9d2f-4212-ad84-7eafa0402c84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0335ef053e5845cd8ead785671a1b0f0, user_id=4e7cbe1bbe5c46e3b224d6bf67d9367a, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:32:43.138495 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b20abfb-9d2f-4212-ad84-7eafa0402c84 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:32:43.211282 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1220/2476] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:32:43 2022] PUT /identity/v3/projects/0335ef053e5845cd8ead785671a1b0f0/users/4e7cbe1bbe5c46e3b224d6bf67d9367a/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:32:43.216765 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9dd31-3582-435e-8662-63d6d7b4d9ec None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:43.217054 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9dd31-3582-435e-8662-63d6d7b4d9ec None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:43.217332 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9dd31-3582-435e-8662-63d6d7b4d9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:43.245357 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-29b9dd31-3582-435e-8662-63d6d7b4d9ec None None] MFA Rules not processed for user `4e7cbe1bbe5c46e3b224d6bf67d9367a`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:32:43.246563 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-29b9dd31-3582-435e-8662-63d6d7b4d9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:32:43.314550 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1257/2477] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:32:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:32:43.319908 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b20abfb-9d2f-4212-ad84-7eafa0402c84 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:32:43.346506 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:32:43.396661 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b20abfb-9d2f-4212-ad84-7eafa0402c84 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:32:43.406082 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6eb34-5a7e-4481-9941-b19d26e6e8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:32:43.406730 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6eb34-5a7e-4481-9941-b19d26e6e8aa admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:43.407013 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6eb34-5a7e-4481-9941-b19d26e6e8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:43.407455 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6eb34-5a7e-4481-9941-b19d26e6e8aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:43.408009 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa6eb34-5a7e-4481-9941-b19d26e6e8aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:32:43.409977 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa6eb34-5a7e-4481-9941-b19d26e6e8aa admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:32:43.510442 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1221/2478] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:32:43 2022] POST /identity/v3/projects => generated 373 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:32:43.514010 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9dd31-3582-435e-8662-63d6d7b4d9ec None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:32:43.542201 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9dd31-3582-435e-8662-63d6d7b4d9ec None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:32:43.558584 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:32:43.617617 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46506927-0145-4e35-9532-49da7cc04485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:32:43.618351 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46506927-0145-4e35-9532-49da7cc04485 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:43.619274 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46506927-0145-4e35-9532-49da7cc04485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:43.619554 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46506927-0145-4e35-9532-49da7cc04485 admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:43.620454 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46506927-0145-4e35-9532-49da7cc04485 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:32:43.622425 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46506927-0145-4e35-9532-49da7cc04485 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:32:43.768072 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1258/2479] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:32:43 2022] POST /identity/v3/users => generated 293 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:32:43.772524 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6eb34-5a7e-4481-9941-b19d26e6e8aa admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:32:43.786575 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6eb34-5a7e-4481-9941-b19d26e6e8aa admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:32:43.791635 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3521fece-d041-403a-9d23-8be1ec8e0ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:32:43.792331 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521fece-d041-403a-9d23-8be1ec8e0ee1 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:43.792840 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521fece-d041-403a-9d23-8be1ec8e0ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:43.793182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521fece-d041-403a-9d23-8be1ec8e0ee1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:43.793981 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3521fece-d041-403a-9d23-8be1ec8e0ee1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:32:43.795703 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3521fece-d041-403a-9d23-8be1ec8e0ee1 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:32:43.808196 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1222/2480] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:32:43.814613 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46506927-0145-4e35-9532-49da7cc04485 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:32:43.842263 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46506927-0145-4e35-9532-49da7cc04485 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:32:43.859545 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec063cd-ad7d-409a-8a73-d6c75a535015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:32:43.859545 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec063cd-ad7d-409a-8a73-d6c75a535015 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:43.859545 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec063cd-ad7d-409a-8a73-d6c75a535015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:43.859545 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec063cd-ad7d-409a-8a73-d6c75a535015 admin admin] PATH_INFO: `/v3/projects/a3ce578e73f74874ad197e88d4e6afd9/users/4f963c8029c24dd5949c5a07e1f75d4a/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:43.870473 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec063cd-ad7d-409a-8a73-d6c75a535015 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3ce578e73f74874ad197e88d4e6afd9, user_id=4f963c8029c24dd5949c5a07e1f75d4a, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:32:43.876280 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec063cd-ad7d-409a-8a73-d6c75a535015 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:32:43.965645 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1259/2481] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:32:43 2022] PUT /identity/v3/projects/a3ce578e73f74874ad197e88d4e6afd9/users/4f963c8029c24dd5949c5a07e1f75d4a/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:32:43.970052 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3521fece-d041-403a-9d23-8be1ec8e0ee1 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:32:43.980551 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3521fece-d041-403a-9d23-8be1ec8e0ee1 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:32:44.041719 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa72454-4475-4102-918b-2d158ed3131e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:32:44.042343 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa72454-4475-4102-918b-2d158ed3131e admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:44.042861 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa72454-4475-4102-918b-2d158ed3131e admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:44.043256 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa72454-4475-4102-918b-2d158ed3131e admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:44.043993 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa72454-4475-4102-918b-2d158ed3131e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:32:44.047272 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa72454-4475-4102-918b-2d158ed3131e admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:32:44.059006 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1223/2482] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:32:44.063088 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec063cd-ad7d-409a-8a73-d6c75a535015 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:32:44.075597 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec063cd-ad7d-409a-8a73-d6c75a535015 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:32:44.080548 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f9146-2fd8-4b9f-bf5b-7ffc51be1357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:32:44.081276 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f9146-2fd8-4b9f-bf5b-7ffc51be1357 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:44.081629 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f9146-2fd8-4b9f-bf5b-7ffc51be1357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:44.081895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f9146-2fd8-4b9f-bf5b-7ffc51be1357 admin admin] PATH_INFO: `/v3/projects/a3ce578e73f74874ad197e88d4e6afd9/users/4f963c8029c24dd5949c5a07e1f75d4a/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:44.084281 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964f9146-2fd8-4b9f-bf5b-7ffc51be1357 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3ce578e73f74874ad197e88d4e6afd9, user_id=4f963c8029c24dd5949c5a07e1f75d4a, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:32:44.087056 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964f9146-2fd8-4b9f-bf5b-7ffc51be1357 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:32:44.099062 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1260/2483] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:32:44 2022] PUT /identity/v3/projects/a3ce578e73f74874ad197e88d4e6afd9/users/4f963c8029c24dd5949c5a07e1f75d4a/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:32:44.109724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4bf366-e8fc-4329-9392-fa3f129b35c2 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:44.110004 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4bf366-e8fc-4329-9392-fa3f129b35c2 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:44.110276 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4bf366-e8fc-4329-9392-fa3f129b35c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:44.141940 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-2f4bf366-e8fc-4329-9392-fa3f129b35c2 None None] MFA Rules not processed for user `4f963c8029c24dd5949c5a07e1f75d4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:32:44.143153 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-2f4bf366-e8fc-4329-9392-fa3f129b35c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:32:44.219132 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1224/2484] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:32:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:32:44.231236 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f9146-2fd8-4b9f-bf5b-7ffc51be1357 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:32:44.232976 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-964f9146-2fd8-4b9f-bf5b-7ffc51be1357 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:32:44.293143 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f9146-2fd8-4b9f-bf5b-7ffc51be1357 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:32:44.296879 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdbc122-dbbd-4d1d-b0a1-9875b79dd35d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:32:44.297511 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbc122-dbbd-4d1d-b0a1-9875b79dd35d service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:44.297773 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbc122-dbbd-4d1d-b0a1-9875b79dd35d service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:44.298042 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbc122-dbbd-4d1d-b0a1-9875b79dd35d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:44.307877 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdbc122-dbbd-4d1d-b0a1-9875b79dd35d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:32:44.309433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdbc122-dbbd-4d1d-b0a1-9875b79dd35d service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:32:44.319498 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1261/2485] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:32:44 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:32:44.516326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4bf366-e8fc-4329-9392-fa3f129b35c2 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:32:44.529031 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4bf366-e8fc-4329-9392-fa3f129b35c2 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:32:44.533129 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45282a0a-2e5d-4e2e-b9fa-8115639e24c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:32:44.533760 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45282a0a-2e5d-4e2e-b9fa-8115639e24c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:32:44.534034 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45282a0a-2e5d-4e2e-b9fa-8115639e24c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:32:44.534614 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45282a0a-2e5d-4e2e-b9fa-8115639e24c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:32:44.550929 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:32:44.598181 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45282a0a-2e5d-4e2e-b9fa-8115639e24c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:32:44.599640 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45282a0a-2e5d-4e2e-b9fa-8115639e24c8 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:32:44.609554 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1225/2486] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:32:44 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 14:32:54.314852 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=62,keystone:INSERT=5 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:32:55.191611 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=20 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:33:05.879002 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdbc122-dbbd-4d1d-b0a1-9875b79dd35d service cinder] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:05.880386 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-1fdbc122-dbbd-4d1d-b0a1-9875b79dd35d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:33:05.952052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdbc122-dbbd-4d1d-b0a1-9875b79dd35d service cinder] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:05.957573 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97b2e58-eb24-4765-9c34-6ee811c711eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:05.958176 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b2e58-eb24-4765-9c34-6ee811c711eb service nova] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:05.958450 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b2e58-eb24-4765-9c34-6ee811c711eb service nova] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:05.958716 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b2e58-eb24-4765-9c34-6ee811c711eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:05.960460 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-a97b2e58-eb24-4765-9c34-6ee811c711eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:33:06.017071 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97b2e58-eb24-4765-9c34-6ee811c711eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:06.018619 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97b2e58-eb24-4765-9c34-6ee811c711eb service nova] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:06.028058 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1262/2487] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:33:05 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:33:16.023284 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=56 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:33:38.962388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45282a0a-2e5d-4e2e-b9fa-8115639e24c8 service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:39.064273 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45282a0a-2e5d-4e2e-b9fa-8115639e24c8 service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:39.069682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6e486-ad5e-4880-b4a2-bd207f8abeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:39.070308 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6e486-ad5e-4880-b4a2-bd207f8abeb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:39.070580 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6e486-ad5e-4880-b4a2-bd207f8abeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:39.070847 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6e486-ad5e-4880-b4a2-bd207f8abeb6 admin admin] PATH_INFO: `/v3/users/4e7cbe1bbe5c46e3b224d6bf67d9367a` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:39.072014 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a6e486-ad5e-4880-b4a2-bd207f8abeb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e7cbe1bbe5c46e3b224d6bf67d9367a)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:39.073971 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a6e486-ad5e-4880-b4a2-bd207f8abeb6 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:39.239756 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-44a6e486-ad5e-4880-b4a2-bd207f8abeb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e7cbe1bbe5c46e3b224d6bf67d9367a'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:39.251773 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-44a6e486-ad5e-4880-b4a2-bd207f8abeb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e7cbe1bbe5c46e3b224d6bf67d9367a'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:39.318485 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-44a6e486-ad5e-4880-b4a2-bd207f8abeb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e7cbe1bbe5c46e3b224d6bf67d9367a'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:39.319087 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-44a6e486-ad5e-4880-b4a2-bd207f8abeb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e7cbe1bbe5c46e3b224d6bf67d9367a'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:39.326192 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-44a6e486-ad5e-4880-b4a2-bd207f8abeb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e7cbe1bbe5c46e3b224d6bf67d9367a'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:39.332006 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1226/2488] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:33:38 2022] DELETE /identity/v3/users/4e7cbe1bbe5c46e3b224d6bf67d9367a => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:33:39.336705 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f66536-30d3-467f-9c56-e3289b7d7f6a None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:39.337192 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f66536-30d3-467f-9c56-e3289b7d7f6a None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:39.337302 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f66536-30d3-467f-9c56-e3289b7d7f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:39.385683 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-b7f66536-30d3-467f-9c56-e3289b7d7f6a None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:33:39.386877 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-b7f66536-30d3-467f-9c56-e3289b7d7f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:33:39.442334 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1263/2489] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:33:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:33:39.451999 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6e486-ad5e-4880-b4a2-bd207f8abeb6 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:39.453399 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-44a6e486-ad5e-4880-b4a2-bd207f8abeb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:33:39.493987 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:33:39.546566 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6e486-ad5e-4880-b4a2-bd207f8abeb6 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:39.550146 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb6180-dec6-4171-9603-db7b40db629d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:39.550899 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb6180-dec6-4171-9603-db7b40db629d service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:39.551182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb6180-dec6-4171-9603-db7b40db629d service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:39.551460 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb6180-dec6-4171-9603-db7b40db629d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:39.560930 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddb6180-dec6-4171-9603-db7b40db629d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:39.562348 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddb6180-dec6-4171-9603-db7b40db629d service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:39.572617 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1227/2490] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:33:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:33:40.009626 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f66536-30d3-467f-9c56-e3289b7d7f6a None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:40.011055 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-b7f66536-30d3-467f-9c56-e3289b7d7f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:33:40.024579 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f66536-30d3-467f-9c56-e3289b7d7f6a None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:40.029400 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63bd392-01d2-42a0-9c86-89a470938a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:40.030108 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63bd392-01d2-42a0-9c86-89a470938a1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:40.030451 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63bd392-01d2-42a0-9c86-89a470938a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:40.030798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63bd392-01d2-42a0-9c86-89a470938a1c admin admin] PATH_INFO: `/v3/projects/0335ef053e5845cd8ead785671a1b0f0` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:40.032028 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63bd392-01d2-42a0-9c86-89a470938a1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0335ef053e5845cd8ead785671a1b0f0)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:40.034063 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63bd392-01d2-42a0-9c86-89a470938a1c admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:40.147276 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-b63bd392-01d2-42a0-9c86-89a470938a1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0335ef053e5845cd8ead785671a1b0f0', 'request_id': 'req-b63bd392-01d2-42a0-9c86-89a470938a1c'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:40.153386 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-b63bd392-01d2-42a0-9c86-89a470938a1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0335ef053e5845cd8ead785671a1b0f0', 'request_id': 'req-b63bd392-01d2-42a0-9c86-89a470938a1c'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:40.158815 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-b63bd392-01d2-42a0-9c86-89a470938a1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0335ef053e5845cd8ead785671a1b0f0', 'request_id': 'req-b63bd392-01d2-42a0-9c86-89a470938a1c'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:40.227408 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-b63bd392-01d2-42a0-9c86-89a470938a1c admin admin] The token cache is being invalidate because project 0335ef053e5845cd8ead785671a1b0f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:33:40.227700 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-b63bd392-01d2-42a0-9c86-89a470938a1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:40.228370 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-b63bd392-01d2-42a0-9c86-89a470938a1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:40.230143 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1264/2491] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:33:40 2022] DELETE /identity/v3/projects/0335ef053e5845cd8ead785671a1b0f0 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:33:40.233962 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb6180-dec6-4171-9603-db7b40db629d service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:40.243967 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb6180-dec6-4171-9603-db7b40db629d service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:40.300806 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5cae7b-1faa-4056-824a-da828bc26ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:40.301366 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5cae7b-1faa-4056-824a-da828bc26ec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:40.301645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5cae7b-1faa-4056-824a-da828bc26ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:40.301916 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5cae7b-1faa-4056-824a-da828bc26ec9 admin admin] PATH_INFO: `/v3/users/4f963c8029c24dd5949c5a07e1f75d4a` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:40.303080 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5cae7b-1faa-4056-824a-da828bc26ec9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f963c8029c24dd5949c5a07e1f75d4a)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:40.305043 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5cae7b-1faa-4056-824a-da828bc26ec9 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:40.463805 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1a5cae7b-1faa-4056-824a-da828bc26ec9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f963c8029c24dd5949c5a07e1f75d4a'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:40.476310 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1a5cae7b-1faa-4056-824a-da828bc26ec9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f963c8029c24dd5949c5a07e1f75d4a'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:40.536717 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1a5cae7b-1faa-4056-824a-da828bc26ec9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f963c8029c24dd5949c5a07e1f75d4a'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:40.537333 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1a5cae7b-1faa-4056-824a-da828bc26ec9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f963c8029c24dd5949c5a07e1f75d4a'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:40.545286 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1a5cae7b-1faa-4056-824a-da828bc26ec9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f963c8029c24dd5949c5a07e1f75d4a'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:40.554781 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1228/2492] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:33:40 2022] DELETE /identity/v3/users/4f963c8029c24dd5949c5a07e1f75d4a => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:33:40.949034 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63bd392-01d2-42a0-9c86-89a470938a1c admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:40.950575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-b63bd392-01d2-42a0-9c86-89a470938a1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:33:40.964481 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63bd392-01d2-42a0-9c86-89a470938a1c admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:41.016883 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:33:41.023314 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea11034-0222-4e3b-b0e6-b14637c8d38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:41.023955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea11034-0222-4e3b-b0e6-b14637c8d38a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:41.024230 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea11034-0222-4e3b-b0e6-b14637c8d38a admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:41.024588 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea11034-0222-4e3b-b0e6-b14637c8d38a admin admin] PATH_INFO: `/v3/projects/a3ce578e73f74874ad197e88d4e6afd9` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:41.029295 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea11034-0222-4e3b-b0e6-b14637c8d38a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3ce578e73f74874ad197e88d4e6afd9)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:41.029295 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea11034-0222-4e3b-b0e6-b14637c8d38a admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:41.122492 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-7ea11034-0222-4e3b-b0e6-b14637c8d38a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3ce578e73f74874ad197e88d4e6afd9', 'request_id': 'req-7ea11034-0222-4e3b-b0e6-b14637c8d38a'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:41.129228 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-7ea11034-0222-4e3b-b0e6-b14637c8d38a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3ce578e73f74874ad197e88d4e6afd9', 'request_id': 'req-7ea11034-0222-4e3b-b0e6-b14637c8d38a'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:41.134829 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-7ea11034-0222-4e3b-b0e6-b14637c8d38a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3ce578e73f74874ad197e88d4e6afd9', 'request_id': 'req-7ea11034-0222-4e3b-b0e6-b14637c8d38a'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:41.225042 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-7ea11034-0222-4e3b-b0e6-b14637c8d38a admin admin] The token cache is being invalidate because project a3ce578e73f74874ad197e88d4e6afd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:33:41.225323 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-7ea11034-0222-4e3b-b0e6-b14637c8d38a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:41.225988 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-7ea11034-0222-4e3b-b0e6-b14637c8d38a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:41.227513 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1265/2493] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:33:40 2022] DELETE /identity/v3/projects/a3ce578e73f74874ad197e88d4e6afd9 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:33:41.232386 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2774f-f548-4ce7-a213-fa9861f01a20 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:41.233717 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2774f-f548-4ce7-a213-fa9861f01a20 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:41.233717 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2774f-f548-4ce7-a213-fa9861f01a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:41.266093 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:33:41.282046 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-5af2774f-f548-4ce7-a213-fa9861f01a20 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:33:41.283289 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5af2774f-f548-4ce7-a213-fa9861f01a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:33:41.343666 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1229/2494] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:33:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:33:41.351595 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ecda7-53db-4a59-b963-462c31d09b24 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:41.351875 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ecda7-53db-4a59-b963-462c31d09b24 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:41.352146 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ecda7-53db-4a59-b963-462c31d09b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:41.372110 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-872ecda7-53db-4a59-b963-462c31d09b24 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:33:41.373387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-872ecda7-53db-4a59-b963-462c31d09b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:33:41.381756 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1266/2495] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:33:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:33:41.385845 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af2774f-f548-4ce7-a213-fa9861f01a20 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:41.395964 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af2774f-f548-4ce7-a213-fa9861f01a20 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:41.400528 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6d565-5c78-41e2-84fb-584db276e55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:41.401127 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6d565-5c78-41e2-84fb-584db276e55d admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:41.401403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6d565-5c78-41e2-84fb-584db276e55d admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:41.401673 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6d565-5c78-41e2-84fb-584db276e55d admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:41.402358 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a6d565-5c78-41e2-84fb-584db276e55d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:33:41.402657 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a6d565-5c78-41e2-84fb-584db276e55d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:41.404262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a6d565-5c78-41e2-84fb-584db276e55d admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:41.415540 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1230/2496] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:33:41.419782 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ecda7-53db-4a59-b963-462c31d09b24 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:41.429480 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ecda7-53db-4a59-b963-462c31d09b24 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:41.434238 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0052fb59-226d-4b8e-b1b0-9c4c7d307ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:41.434845 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052fb59-226d-4b8e-b1b0-9c4c7d307ea8 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:41.435123 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052fb59-226d-4b8e-b1b0-9c4c7d307ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:41.435394 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052fb59-226d-4b8e-b1b0-9c4c7d307ea8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:41.436087 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0052fb59-226d-4b8e-b1b0-9c4c7d307ea8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:41.438021 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0052fb59-226d-4b8e-b1b0-9c4c7d307ea8 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:41.549046 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1267/2497] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:33:41 2022] POST /identity/v3/projects => generated 361 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:33:41.553641 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6d565-5c78-41e2-84fb-584db276e55d admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:41.563510 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6d565-5c78-41e2-84fb-584db276e55d admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:41.619833 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ef639-e65b-4ba1-865b-8302bde03a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:41.620459 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ef639-e65b-4ba1-865b-8302bde03a38 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:41.620743 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ef639-e65b-4ba1-865b-8302bde03a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:41.621018 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ef639-e65b-4ba1-865b-8302bde03a38 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:41.621680 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ef639-e65b-4ba1-865b-8302bde03a38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:41.623642 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ef639-e65b-4ba1-865b-8302bde03a38 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:41.724354 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1231/2498] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:33:41 2022] POST /identity/v3/users => generated 281 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:33:41.728500 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0052fb59-226d-4b8e-b1b0-9c4c7d307ea8 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:41.739139 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0052fb59-226d-4b8e-b1b0-9c4c7d307ea8 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:41.743760 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff1fc7-9702-4af4-b60e-e353536fdf8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:41.744364 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff1fc7-9702-4af4-b60e-e353536fdf8d admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:41.744673 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff1fc7-9702-4af4-b60e-e353536fdf8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:41.744946 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff1fc7-9702-4af4-b60e-e353536fdf8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:41.745631 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff1fc7-9702-4af4-b60e-e353536fdf8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:41.747327 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff1fc7-9702-4af4-b60e-e353536fdf8d admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:41.758036 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1268/2499] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:33:41.762105 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ef639-e65b-4ba1-865b-8302bde03a38 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:41.772136 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ef639-e65b-4ba1-865b-8302bde03a38 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:41.776701 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0853a-c550-4aee-ba95-38027de409d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:41.777290 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0853a-c550-4aee-ba95-38027de409d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:41.777563 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0853a-c550-4aee-ba95-38027de409d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:41.777828 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0853a-c550-4aee-ba95-38027de409d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:41.778505 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d0853a-c550-4aee-ba95-38027de409d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:41.780059 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d0853a-c550-4aee-ba95-38027de409d3 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:41.792663 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1232/2500] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:33:41.796656 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff1fc7-9702-4af4-b60e-e353536fdf8d admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:41.807062 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff1fc7-9702-4af4-b60e-e353536fdf8d admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:41.811552 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4704ab-9abf-49e4-bfee-9cd2664f354e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:41.812191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4704ab-9abf-49e4-bfee-9cd2664f354e admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:41.812496 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4704ab-9abf-49e4-bfee-9cd2664f354e admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:41.812824 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4704ab-9abf-49e4-bfee-9cd2664f354e admin admin] PATH_INFO: `/v3/projects/e0f25d1b899747ca9010e27008f82225/users/41791a0303f347ee9779180e701b9375/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:41.827348 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4704ab-9abf-49e4-bfee-9cd2664f354e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0f25d1b899747ca9010e27008f82225, user_id=41791a0303f347ee9779180e701b9375, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:41.830026 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4704ab-9abf-49e4-bfee-9cd2664f354e admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:41.946541 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1269/2501] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:33:41 2022] PUT /identity/v3/projects/e0f25d1b899747ca9010e27008f82225/users/41791a0303f347ee9779180e701b9375/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:33:41.956849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7af03-3ee7-4269-b2cf-32a28d878ed8 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:41.957144 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7af03-3ee7-4269-b2cf-32a28d878ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:41.957441 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7af03-3ee7-4269-b2cf-32a28d878ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:41.985709 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-46d7af03-3ee7-4269-b2cf-32a28d878ed8 None None] MFA Rules not processed for user `41791a0303f347ee9779180e701b9375`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:33:41.986916 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-46d7af03-3ee7-4269-b2cf-32a28d878ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:33:42.042863 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:33:42.052214 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1233/2502] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:33:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:33:42.056844 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4704ab-9abf-49e4-bfee-9cd2664f354e admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:42.117091 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4704ab-9abf-49e4-bfee-9cd2664f354e admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:42.121599 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33097148-db9e-4147-8d3b-0fce21097611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:42.122202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33097148-db9e-4147-8d3b-0fce21097611 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:42.122480 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33097148-db9e-4147-8d3b-0fce21097611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:42.122751 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33097148-db9e-4147-8d3b-0fce21097611 admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:42.123399 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33097148-db9e-4147-8d3b-0fce21097611 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:42.125322 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33097148-db9e-4147-8d3b-0fce21097611 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:42.227686 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1270/2503] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:33:42 2022] POST /identity/v3/projects => generated 365 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:33:42.231763 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d7af03-3ee7-4269-b2cf-32a28d878ed8 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:42.242552 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d7af03-3ee7-4269-b2cf-32a28d878ed8 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:42.299035 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10076a-82f0-4584-bf54-4d5273052c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:42.299702 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10076a-82f0-4584-bf54-4d5273052c06 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:42.299915 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10076a-82f0-4584-bf54-4d5273052c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:42.300184 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10076a-82f0-4584-bf54-4d5273052c06 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:42.300881 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce10076a-82f0-4584-bf54-4d5273052c06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:42.302908 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce10076a-82f0-4584-bf54-4d5273052c06 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:42.368531 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1234/2504] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:33:42 2022] POST /identity/v3/users => generated 289 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:33:42.372670 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33097148-db9e-4147-8d3b-0fce21097611 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:42.383141 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33097148-db9e-4147-8d3b-0fce21097611 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:42.387754 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d77ab-5902-481b-a78e-3374d36b2658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:42.388348 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d77ab-5902-481b-a78e-3374d36b2658 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:42.388655 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d77ab-5902-481b-a78e-3374d36b2658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:42.388926 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d77ab-5902-481b-a78e-3374d36b2658 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:42.389608 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0d77ab-5902-481b-a78e-3374d36b2658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:42.391178 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0d77ab-5902-481b-a78e-3374d36b2658 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:42.393269 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:33:42.402583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1271/2505] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:33:42.406594 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10076a-82f0-4584-bf54-4d5273052c06 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:42.417101 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10076a-82f0-4584-bf54-4d5273052c06 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:42.422631 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5f9e46-b927-4d84-ae74-8297f8b92d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:42.422631 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f9e46-b927-4d84-ae74-8297f8b92d78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:42.422631 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f9e46-b927-4d84-ae74-8297f8b92d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:42.422955 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f9e46-b927-4d84-ae74-8297f8b92d78 admin admin] PATH_INFO: `/v3/projects/957b43c77a754b6d9f15f4812ead1ceb/users/aecf657f884a4f55829085085b482036/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:42.437268 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5f9e46-b927-4d84-ae74-8297f8b92d78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=957b43c77a754b6d9f15f4812ead1ceb, user_id=aecf657f884a4f55829085085b482036, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:42.439903 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5f9e46-b927-4d84-ae74-8297f8b92d78 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:42.507918 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1235/2506] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:33:42 2022] PUT /identity/v3/projects/957b43c77a754b6d9f15f4812ead1ceb/users/aecf657f884a4f55829085085b482036/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:33:42.511985 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d77ab-5902-481b-a78e-3374d36b2658 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:42.525520 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d77ab-5902-481b-a78e-3374d36b2658 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:42.579588 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba387516-52da-40aa-ad67-074492c169c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:42.580192 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba387516-52da-40aa-ad67-074492c169c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:42.580562 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba387516-52da-40aa-ad67-074492c169c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:42.580837 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba387516-52da-40aa-ad67-074492c169c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:42.581514 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba387516-52da-40aa-ad67-074492c169c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:42.584248 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba387516-52da-40aa-ad67-074492c169c5 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:42.593974 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1272/2507] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:33:42.598579 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5f9e46-b927-4d84-ae74-8297f8b92d78 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:42.608596 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5f9e46-b927-4d84-ae74-8297f8b92d78 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:42.613112 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244edc17-7251-4c35-9901-1dc868a479f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:42.614110 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244edc17-7251-4c35-9901-1dc868a479f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:42.614386 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244edc17-7251-4c35-9901-1dc868a479f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:42.614656 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244edc17-7251-4c35-9901-1dc868a479f4 admin admin] PATH_INFO: `/v3/projects/957b43c77a754b6d9f15f4812ead1ceb/users/aecf657f884a4f55829085085b482036/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:42.616628 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244edc17-7251-4c35-9901-1dc868a479f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=957b43c77a754b6d9f15f4812ead1ceb, user_id=aecf657f884a4f55829085085b482036, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:42.619233 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244edc17-7251-4c35-9901-1dc868a479f4 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:42.630249 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1236/2508] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:33:42 2022] PUT /identity/v3/projects/957b43c77a754b6d9f15f4812ead1ceb/users/aecf657f884a4f55829085085b482036/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:33:42.641009 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e7823-0928-4f74-b469-4a7755f46220 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:42.641287 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e7823-0928-4f74-b469-4a7755f46220 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:42.641557 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e7823-0928-4f74-b469-4a7755f46220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:42.669413 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-b36e7823-0928-4f74-b469-4a7755f46220 None None] MFA Rules not processed for user `aecf657f884a4f55829085085b482036`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:33:42.670601 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-b36e7823-0928-4f74-b469-4a7755f46220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:33:42.737703 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1273/2509] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:33:42 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:33:42.749455 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244edc17-7251-4c35-9901-1dc868a479f4 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:42.751169 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-244edc17-7251-4c35-9901-1dc868a479f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:33:42.828506 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244edc17-7251-4c35-9901-1dc868a479f4 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:42.833085 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fc23a7-dbd7-4f30-9661-2fb93aac66a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:42.833589 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc23a7-dbd7-4f30-9661-2fb93aac66a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:42.833873 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc23a7-dbd7-4f30-9661-2fb93aac66a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:42.834145 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc23a7-dbd7-4f30-9661-2fb93aac66a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:42.843470 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fc23a7-dbd7-4f30-9661-2fb93aac66a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:42.844972 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fc23a7-dbd7-4f30-9661-2fb93aac66a0 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:42.854990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1237/2510] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:33:42 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:33:43.064100 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e7823-0928-4f74-b469-4a7755f46220 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:43.073929 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e7823-0928-4f74-b469-4a7755f46220 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:43.077532 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232d65e-0678-4579-98b4-9f6940e3eaff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:43.078174 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232d65e-0678-4579-98b4-9f6940e3eaff service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:43.078453 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232d65e-0678-4579-98b4-9f6940e3eaff service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:43.078741 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232d65e-0678-4579-98b4-9f6940e3eaff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:43.133993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6232d65e-0678-4579-98b4-9f6940e3eaff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:43.135436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6232d65e-0678-4579-98b4-9f6940e3eaff service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:43.145309 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1274/2511] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:33:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:33:46.459767 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fc23a7-dbd7-4f30-9661-2fb93aac66a0 service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:46.492906 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:33:46.521609 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fc23a7-dbd7-4f30-9661-2fb93aac66a0 service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:46.527037 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ee17a-4a3a-49c7-ace8-defc8a54263e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:46.527673 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ee17a-4a3a-49c7-ace8-defc8a54263e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:46.527950 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ee17a-4a3a-49c7-ace8-defc8a54263e admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:46.528220 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ee17a-4a3a-49c7-ace8-defc8a54263e admin admin] PATH_INFO: `/v3/users/41791a0303f347ee9779180e701b9375` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:46.529433 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ee17a-4a3a-49c7-ace8-defc8a54263e admin admin] RBAC: Authorizing `identity:delete_user(user_id=41791a0303f347ee9779180e701b9375)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:46.531337 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ee17a-4a3a-49c7-ace8-defc8a54263e admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:46.696626 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c29ee17a-4a3a-49c7-ace8-defc8a54263e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41791a0303f347ee9779180e701b9375'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:46.708915 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c29ee17a-4a3a-49c7-ace8-defc8a54263e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41791a0303f347ee9779180e701b9375'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:46.780199 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c29ee17a-4a3a-49c7-ace8-defc8a54263e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41791a0303f347ee9779180e701b9375'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:46.780810 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c29ee17a-4a3a-49c7-ace8-defc8a54263e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41791a0303f347ee9779180e701b9375'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:46.788678 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c29ee17a-4a3a-49c7-ace8-defc8a54263e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41791a0303f347ee9779180e701b9375'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:46.794601 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1238/2512] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:33:46 2022] DELETE /identity/v3/users/41791a0303f347ee9779180e701b9375 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:33:46.799074 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7418ce-414e-473a-991f-ca1873f06f46 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:46.799558 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7418ce-414e-473a-991f-ca1873f06f46 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:46.799674 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7418ce-414e-473a-991f-ca1873f06f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:46.819369 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-ef7418ce-414e-473a-991f-ca1873f06f46 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:33:46.820623 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-ef7418ce-414e-473a-991f-ca1873f06f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:33:46.832636 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:33:46.878867 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1275/2513] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:33:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:33:46.888551 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ee17a-4a3a-49c7-ace8-defc8a54263e admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:46.889942 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c29ee17a-4a3a-49c7-ace8-defc8a54263e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:33:46.946061 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ee17a-4a3a-49c7-ace8-defc8a54263e admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:46.949750 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03251ed5-e89a-459b-a218-c14cee8c9521 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:46.950361 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03251ed5-e89a-459b-a218-c14cee8c9521 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:46.950640 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03251ed5-e89a-459b-a218-c14cee8c9521 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:46.950912 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03251ed5-e89a-459b-a218-c14cee8c9521 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:46.960514 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03251ed5-e89a-459b-a218-c14cee8c9521 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:46.961936 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03251ed5-e89a-459b-a218-c14cee8c9521 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:46.971877 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1239/2514] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:33:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:33:47.377571 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7418ce-414e-473a-991f-ca1873f06f46 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:47.379549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-ef7418ce-414e-473a-991f-ca1873f06f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:33:47.391497 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7418ce-414e-473a-991f-ca1873f06f46 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:47.396809 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9441adf2-d5a2-4045-ab8d-80aeb4a9be2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:47.397452 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441adf2-d5a2-4045-ab8d-80aeb4a9be2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:47.397724 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441adf2-d5a2-4045-ab8d-80aeb4a9be2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:47.397992 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441adf2-d5a2-4045-ab8d-80aeb4a9be2b admin admin] PATH_INFO: `/v3/projects/e0f25d1b899747ca9010e27008f82225` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:47.399135 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9441adf2-d5a2-4045-ab8d-80aeb4a9be2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0f25d1b899747ca9010e27008f82225)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:47.405446 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9441adf2-d5a2-4045-ab8d-80aeb4a9be2b admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:47.537183 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9441adf2-d5a2-4045-ab8d-80aeb4a9be2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0f25d1b899747ca9010e27008f82225', 'request_id': 'req-9441adf2-d5a2-4045-ab8d-80aeb4a9be2b'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:47.544961 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9441adf2-d5a2-4045-ab8d-80aeb4a9be2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0f25d1b899747ca9010e27008f82225', 'request_id': 'req-9441adf2-d5a2-4045-ab8d-80aeb4a9be2b'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:47.552781 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9441adf2-d5a2-4045-ab8d-80aeb4a9be2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0f25d1b899747ca9010e27008f82225', 'request_id': 'req-9441adf2-d5a2-4045-ab8d-80aeb4a9be2b'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:47.609696 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9441adf2-d5a2-4045-ab8d-80aeb4a9be2b admin admin] The token cache is being invalidate because project e0f25d1b899747ca9010e27008f82225 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:33:47.609978 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9441adf2-d5a2-4045-ab8d-80aeb4a9be2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:47.610595 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9441adf2-d5a2-4045-ab8d-80aeb4a9be2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:47.612099 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1276/2515] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:33:47 2022] DELETE /identity/v3/projects/e0f25d1b899747ca9010e27008f82225 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:33:47.620764 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03251ed5-e89a-459b-a218-c14cee8c9521 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:47.631323 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03251ed5-e89a-459b-a218-c14cee8c9521 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:47.695197 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:33:47.708589 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e087633d-5421-4108-bf92-d3bb5a41c674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:47.710351 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087633d-5421-4108-bf92-d3bb5a41c674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:47.711577 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087633d-5421-4108-bf92-d3bb5a41c674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:47.711858 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087633d-5421-4108-bf92-d3bb5a41c674 admin admin] PATH_INFO: `/v3/users/aecf657f884a4f55829085085b482036` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:47.713035 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e087633d-5421-4108-bf92-d3bb5a41c674 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aecf657f884a4f55829085085b482036)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:47.716118 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e087633d-5421-4108-bf92-d3bb5a41c674 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:47.949538 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e087633d-5421-4108-bf92-d3bb5a41c674 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aecf657f884a4f55829085085b482036'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:47.965818 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e087633d-5421-4108-bf92-d3bb5a41c674 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aecf657f884a4f55829085085b482036'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:48.044567 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e087633d-5421-4108-bf92-d3bb5a41c674 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aecf657f884a4f55829085085b482036'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:48.045185 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e087633d-5421-4108-bf92-d3bb5a41c674 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aecf657f884a4f55829085085b482036'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:48.055100 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e087633d-5421-4108-bf92-d3bb5a41c674 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aecf657f884a4f55829085085b482036'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:48.061314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1240/2516] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:33:47 2022] DELETE /identity/v3/users/aecf657f884a4f55829085085b482036 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:33:48.512042 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9441adf2-d5a2-4045-ab8d-80aeb4a9be2b admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:48.513643 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-9441adf2-d5a2-4045-ab8d-80aeb4a9be2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:33:48.539300 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9441adf2-d5a2-4045-ab8d-80aeb4a9be2b admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:48.610451 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9462fa-baf4-4376-a33d-3b9d912804b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:48.611103 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9462fa-baf4-4376-a33d-3b9d912804b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:48.611379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9462fa-baf4-4376-a33d-3b9d912804b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:48.615803 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9462fa-baf4-4376-a33d-3b9d912804b0 admin admin] PATH_INFO: `/v3/projects/957b43c77a754b6d9f15f4812ead1ceb` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:48.615803 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9462fa-baf4-4376-a33d-3b9d912804b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=957b43c77a754b6d9f15f4812ead1ceb)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:48.615803 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9462fa-baf4-4376-a33d-3b9d912804b0 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:48.749623 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:33:48.753215 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9d9462fa-baf4-4376-a33d-3b9d912804b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '957b43c77a754b6d9f15f4812ead1ceb', 'request_id': 'req-9d9462fa-baf4-4376-a33d-3b9d912804b0'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:48.759541 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9d9462fa-baf4-4376-a33d-3b9d912804b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '957b43c77a754b6d9f15f4812ead1ceb', 'request_id': 'req-9d9462fa-baf4-4376-a33d-3b9d912804b0'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:48.765020 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9d9462fa-baf4-4376-a33d-3b9d912804b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '957b43c77a754b6d9f15f4812ead1ceb', 'request_id': 'req-9d9462fa-baf4-4376-a33d-3b9d912804b0'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:48.855006 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9d9462fa-baf4-4376-a33d-3b9d912804b0 admin admin] The token cache is being invalidate because project 957b43c77a754b6d9f15f4812ead1ceb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:33:48.855294 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9d9462fa-baf4-4376-a33d-3b9d912804b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:48.855911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-9d9462fa-baf4-4376-a33d-3b9d912804b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:33:48.857479 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1277/2517] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:33:48 2022] DELETE /identity/v3/projects/957b43c77a754b6d9f15f4812ead1ceb => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:33:48.862476 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f929354-5167-490e-8b98-1694059f65a5 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:48.862975 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f929354-5167-490e-8b98-1694059f65a5 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:48.863072 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f929354-5167-490e-8b98-1694059f65a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:48.883404 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-7f929354-5167-490e-8b98-1694059f65a5 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:33:48.884638 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-7f929354-5167-490e-8b98-1694059f65a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:33:48.949865 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1241/2518] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:33:48.958199 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d25d11-06fc-4e15-93a9-5da963ff827e None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:48.958480 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d25d11-06fc-4e15-93a9-5da963ff827e None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:48.958758 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d25d11-06fc-4e15-93a9-5da963ff827e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:48.979499 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-91d25d11-06fc-4e15-93a9-5da963ff827e None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:33:48.980724 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-91d25d11-06fc-4e15-93a9-5da963ff827e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:33:48.988512 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1278/2519] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:33:48.992718 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f929354-5167-490e-8b98-1694059f65a5 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:49.005780 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f929354-5167-490e-8b98-1694059f65a5 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:49.007207 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51b4a3-37d9-4835-8234-63216ea65ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:49.007838 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51b4a3-37d9-4835-8234-63216ea65ca4 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:49.008115 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51b4a3-37d9-4835-8234-63216ea65ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:49.008385 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51b4a3-37d9-4835-8234-63216ea65ca4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:49.009105 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e51b4a3-37d9-4835-8234-63216ea65ca4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:33:49.009431 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e51b4a3-37d9-4835-8234-63216ea65ca4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:49.011036 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e51b4a3-37d9-4835-8234-63216ea65ca4 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:49.023007 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1242/2520] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:33:49.027234 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d25d11-06fc-4e15-93a9-5da963ff827e None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:49.036923 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d25d11-06fc-4e15-93a9-5da963ff827e None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:49.041492 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d65ed3-d089-49a0-a8db-779795337722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:49.042107 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d65ed3-d089-49a0-a8db-779795337722 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:49.042378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d65ed3-d089-49a0-a8db-779795337722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:49.042644 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d65ed3-d089-49a0-a8db-779795337722 admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:49.043296 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d65ed3-d089-49a0-a8db-779795337722 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:49.045199 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d65ed3-d089-49a0-a8db-779795337722 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:49.108157 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1279/2521] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:33:49 2022] POST /identity/v3/projects => generated 387 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:33:49.112278 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51b4a3-37d9-4835-8234-63216ea65ca4 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:49.122636 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51b4a3-37d9-4835-8234-63216ea65ca4 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:49.178370 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd99bde1-22dc-430f-a4ac-4c06abfb1fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:49.178971 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99bde1-22dc-430f-a4ac-4c06abfb1fc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:49.179244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99bde1-22dc-430f-a4ac-4c06abfb1fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:49.179512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99bde1-22dc-430f-a4ac-4c06abfb1fc9 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:49.180182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd99bde1-22dc-430f-a4ac-4c06abfb1fc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:49.182151 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd99bde1-22dc-430f-a4ac-4c06abfb1fc9 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:49.189854 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:33:49.273924 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1243/2522] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:33:49 2022] POST /identity/v3/users => generated 294 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:33:49.278081 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d65ed3-d089-49a0-a8db-779795337722 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:49.288305 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d65ed3-d089-49a0-a8db-779795337722 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:49.293002 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83506e31-44c6-4c6f-b7a1-21f91a5615fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:49.293600 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83506e31-44c6-4c6f-b7a1-21f91a5615fb admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:49.293874 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83506e31-44c6-4c6f-b7a1-21f91a5615fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:49.294158 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83506e31-44c6-4c6f-b7a1-21f91a5615fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:49.294814 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83506e31-44c6-4c6f-b7a1-21f91a5615fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:49.296496 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83506e31-44c6-4c6f-b7a1-21f91a5615fb admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:49.306306 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1280/2523] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:33:49.315453 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd99bde1-22dc-430f-a4ac-4c06abfb1fc9 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:49.326112 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd99bde1-22dc-430f-a4ac-4c06abfb1fc9 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:49.331265 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd528c0-1d0a-4699-9c18-83ea9a8529a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:49.331870 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd528c0-1d0a-4699-9c18-83ea9a8529a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:49.332147 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd528c0-1d0a-4699-9c18-83ea9a8529a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:49.332447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd528c0-1d0a-4699-9c18-83ea9a8529a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:49.333159 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd528c0-1d0a-4699-9c18-83ea9a8529a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:49.335430 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd528c0-1d0a-4699-9c18-83ea9a8529a1 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:49.346283 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1244/2524] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:33:49.350357 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83506e31-44c6-4c6f-b7a1-21f91a5615fb admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:49.360120 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83506e31-44c6-4c6f-b7a1-21f91a5615fb admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:49.364630 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689874bf-9cf5-4b66-9dbd-2b0565cd0017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:49.365435 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689874bf-9cf5-4b66-9dbd-2b0565cd0017 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:49.365710 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689874bf-9cf5-4b66-9dbd-2b0565cd0017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:49.365982 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689874bf-9cf5-4b66-9dbd-2b0565cd0017 admin admin] PATH_INFO: `/v3/projects/1506f697b12745a9a3dcec792da6d20c/users/9e2929a96b424fde8f63e763f9610191/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:49.380731 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689874bf-9cf5-4b66-9dbd-2b0565cd0017 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1506f697b12745a9a3dcec792da6d20c, user_id=9e2929a96b424fde8f63e763f9610191, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:49.383778 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689874bf-9cf5-4b66-9dbd-2b0565cd0017 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:49.455499 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1281/2525] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:33:49 2022] PUT /identity/v3/projects/1506f697b12745a9a3dcec792da6d20c/users/9e2929a96b424fde8f63e763f9610191/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:33:49.466558 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91ceeb-7459-4ad9-b401-d6dcde5b715d None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:49.466841 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91ceeb-7459-4ad9-b401-d6dcde5b715d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:49.467110 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91ceeb-7459-4ad9-b401-d6dcde5b715d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:49.495123 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-ab91ceeb-7459-4ad9-b401-d6dcde5b715d None None] MFA Rules not processed for user `9e2929a96b424fde8f63e763f9610191`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:33:49.496342 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ab91ceeb-7459-4ad9-b401-d6dcde5b715d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:33:49.559582 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1245/2526] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:33:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:33:49.564097 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689874bf-9cf5-4b66-9dbd-2b0565cd0017 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:49.626330 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689874bf-9cf5-4b66-9dbd-2b0565cd0017 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:49.631469 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574b6e1-e134-4191-a0c3-602b7a91fd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:49.632082 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574b6e1-e134-4191-a0c3-602b7a91fd9c admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:49.632361 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574b6e1-e134-4191-a0c3-602b7a91fd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:49.632664 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574b6e1-e134-4191-a0c3-602b7a91fd9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:49.633434 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4574b6e1-e134-4191-a0c3-602b7a91fd9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:49.635349 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4574b6e1-e134-4191-a0c3-602b7a91fd9c admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:49.777734 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1282/2527] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:33:49 2022] POST /identity/v3/projects => generated 387 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:33:49.781741 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab91ceeb-7459-4ad9-b401-d6dcde5b715d None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:49.793607 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab91ceeb-7459-4ad9-b401-d6dcde5b715d None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:49.850824 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9cdc95-2f11-48f0-8d20-b510dfeecb98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:49.851440 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9cdc95-2f11-48f0-8d20-b510dfeecb98 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:49.851720 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9cdc95-2f11-48f0-8d20-b510dfeecb98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:49.851996 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9cdc95-2f11-48f0-8d20-b510dfeecb98 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:49.852701 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9cdc95-2f11-48f0-8d20-b510dfeecb98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:49.854642 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9cdc95-2f11-48f0-8d20-b510dfeecb98 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:49.934419 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1246/2528] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:33:49 2022] POST /identity/v3/users => generated 294 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:33:49.938490 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574b6e1-e134-4191-a0c3-602b7a91fd9c admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:49.948916 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574b6e1-e134-4191-a0c3-602b7a91fd9c admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:49.953522 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81609d-8d61-4ba4-b96c-6447540f4d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:49.954125 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81609d-8d61-4ba4-b96c-6447540f4d9f admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:49.954405 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81609d-8d61-4ba4-b96c-6447540f4d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:49.954679 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81609d-8d61-4ba4-b96c-6447540f4d9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:49.955386 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d81609d-8d61-4ba4-b96c-6447540f4d9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:49.957056 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d81609d-8d61-4ba4-b96c-6447540f4d9f admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:49.968509 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1283/2529] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:33:49.972516 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9cdc95-2f11-48f0-8d20-b510dfeecb98 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:49.983384 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9cdc95-2f11-48f0-8d20-b510dfeecb98 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:49.988010 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87def370-91eb-482a-8afd-44ccf553298c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:49.988657 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87def370-91eb-482a-8afd-44ccf553298c admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:49.988938 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87def370-91eb-482a-8afd-44ccf553298c admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:49.989215 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87def370-91eb-482a-8afd-44ccf553298c admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:49.989945 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87def370-91eb-482a-8afd-44ccf553298c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:49.991539 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87def370-91eb-482a-8afd-44ccf553298c admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:50.001410 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1247/2530] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:33:50.006204 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81609d-8d61-4ba4-b96c-6447540f4d9f admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:50.016039 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81609d-8d61-4ba4-b96c-6447540f4d9f admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:50.020866 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d4dc39-0421-40b1-89ae-a87105fcf14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:50.022126 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4dc39-0421-40b1-89ae-a87105fcf14f admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:50.022399 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4dc39-0421-40b1-89ae-a87105fcf14f admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:50.022678 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4dc39-0421-40b1-89ae-a87105fcf14f admin admin] PATH_INFO: `/v3/projects/d8f4d5e128524f1bb0f1905480605d49/users/d21a2d43eafb44e98f7483ae0c57bf92/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:50.037216 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d4dc39-0421-40b1-89ae-a87105fcf14f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8f4d5e128524f1bb0f1905480605d49, user_id=d21a2d43eafb44e98f7483ae0c57bf92, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:50.039779 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d4dc39-0421-40b1-89ae-a87105fcf14f admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:50.128754 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1284/2531] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:33:50 2022] PUT /identity/v3/projects/d8f4d5e128524f1bb0f1905480605d49/users/d21a2d43eafb44e98f7483ae0c57bf92/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:33:50.140901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbc405-c474-47a4-ac41-420202473187 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:50.141196 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbc405-c474-47a4-ac41-420202473187 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:50.141462 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbc405-c474-47a4-ac41-420202473187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:50.170596 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-b2bbc405-c474-47a4-ac41-420202473187 None None] MFA Rules not processed for user `d21a2d43eafb44e98f7483ae0c57bf92`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:33:50.171793 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-b2bbc405-c474-47a4-ac41-420202473187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:33:50.205651 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:33:50.238202 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1248/2532] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:33:50 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:33:50.242816 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d4dc39-0421-40b1-89ae-a87105fcf14f admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:50.262850 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:33:50.304510 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d4dc39-0421-40b1-89ae-a87105fcf14f admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:50.309072 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9942d84-7390-43ed-88c9-52b7911e2661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:50.309677 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9942d84-7390-43ed-88c9-52b7911e2661 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:50.309949 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9942d84-7390-43ed-88c9-52b7911e2661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:50.310212 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9942d84-7390-43ed-88c9-52b7911e2661 admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:50.310875 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9942d84-7390-43ed-88c9-52b7911e2661 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:50.312775 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9942d84-7390-43ed-88c9-52b7911e2661 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:50.422953 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1285/2533] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:33:50 2022] POST /identity/v3/projects => generated 387 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:33:50.427990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bbc405-c474-47a4-ac41-420202473187 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:50.438947 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bbc405-c474-47a4-ac41-420202473187 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:50.495183 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f9448c-c1d2-4df7-9523-35cd5987b386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:50.495794 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9448c-c1d2-4df7-9523-35cd5987b386 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:50.496073 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9448c-c1d2-4df7-9523-35cd5987b386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:50.496389 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9448c-c1d2-4df7-9523-35cd5987b386 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:50.497075 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f9448c-c1d2-4df7-9523-35cd5987b386 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:50.499016 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f9448c-c1d2-4df7-9523-35cd5987b386 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:50.585857 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1249/2534] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:33:50 2022] POST /identity/v3/users => generated 300 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:33:50.587052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9942d84-7390-43ed-88c9-52b7911e2661 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:50.599014 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9942d84-7390-43ed-88c9-52b7911e2661 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:50.603725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f12b49-308b-42ff-9d14-6649bacbaca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:50.604357 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f12b49-308b-42ff-9d14-6649bacbaca4 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:50.604655 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f12b49-308b-42ff-9d14-6649bacbaca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:50.604929 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f12b49-308b-42ff-9d14-6649bacbaca4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:50.605651 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f12b49-308b-42ff-9d14-6649bacbaca4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:50.607218 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f12b49-308b-42ff-9d14-6649bacbaca4 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:50.618164 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1286/2535] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:33:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:33:50.622431 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f9448c-c1d2-4df7-9523-35cd5987b386 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:50.633525 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f9448c-c1d2-4df7-9523-35cd5987b386 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:50.638239 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2a032-67a2-4561-ab9e-67ea70e20747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:50.638951 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2a032-67a2-4561-ab9e-67ea70e20747 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:50.639247 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2a032-67a2-4561-ab9e-67ea70e20747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:50.639524 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2a032-67a2-4561-ab9e-67ea70e20747 admin admin] PATH_INFO: `/v3/projects/9d5e81234b9d4e71af079fb3049c041f/users/ca6c80c6e5264437a62e7667a0279f14/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:50.655430 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2a032-67a2-4561-ab9e-67ea70e20747 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d5e81234b9d4e71af079fb3049c041f, user_id=ca6c80c6e5264437a62e7667a0279f14, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:50.658383 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2a032-67a2-4561-ab9e-67ea70e20747 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:50.715263 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1250/2536] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:33:50 2022] PUT /identity/v3/projects/9d5e81234b9d4e71af079fb3049c041f/users/ca6c80c6e5264437a62e7667a0279f14/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:33:50.719663 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f12b49-308b-42ff-9d14-6649bacbaca4 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:50.731717 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f12b49-308b-42ff-9d14-6649bacbaca4 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:50.808314 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522cc0a5-664b-48fc-87b3-e54ac1841120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:50.808962 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522cc0a5-664b-48fc-87b3-e54ac1841120 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:50.809236 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522cc0a5-664b-48fc-87b3-e54ac1841120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:50.809503 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522cc0a5-664b-48fc-87b3-e54ac1841120 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:50.810225 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522cc0a5-664b-48fc-87b3-e54ac1841120 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:50.811794 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522cc0a5-664b-48fc-87b3-e54ac1841120 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:50.824809 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1287/2537] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:33:50 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:33:50.828966 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2a032-67a2-4561-ab9e-67ea70e20747 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:50.841466 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2a032-67a2-4561-ab9e-67ea70e20747 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:50.846690 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e0a1e3-2176-47d2-8d34-532c472de6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:50.847378 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0a1e3-2176-47d2-8d34-532c472de6f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:50.848000 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0a1e3-2176-47d2-8d34-532c472de6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:50.848000 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0a1e3-2176-47d2-8d34-532c472de6f1 admin admin] PATH_INFO: `/v3/projects/9d5e81234b9d4e71af079fb3049c041f/users/ca6c80c6e5264437a62e7667a0279f14/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:50.850806 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e0a1e3-2176-47d2-8d34-532c472de6f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d5e81234b9d4e71af079fb3049c041f, user_id=ca6c80c6e5264437a62e7667a0279f14, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:50.853489 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e0a1e3-2176-47d2-8d34-532c472de6f1 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:50.865465 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1251/2538] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:33:50 2022] PUT /identity/v3/projects/9d5e81234b9d4e71af079fb3049c041f/users/ca6c80c6e5264437a62e7667a0279f14/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:33:50.877791 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6cb43f-89f6-4b5a-a875-ae2155aed40f None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:50.881172 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6cb43f-89f6-4b5a-a875-ae2155aed40f None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:50.881172 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6cb43f-89f6-4b5a-a875-ae2155aed40f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:50.906737 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-1c6cb43f-89f6-4b5a-a875-ae2155aed40f None None] MFA Rules not processed for user `ca6c80c6e5264437a62e7667a0279f14`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:33:50.907937 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-1c6cb43f-89f6-4b5a-a875-ae2155aed40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:33:50.976180 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1288/2539] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:33:50 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:33:50.987070 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e0a1e3-2176-47d2-8d34-532c472de6f1 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:50.988829 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-50e0a1e3-2176-47d2-8d34-532c472de6f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:33:51.046281 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e0a1e3-2176-47d2-8d34-532c472de6f1 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:51.049972 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa44a549-db5b-46ca-8e81-6cb44c898f6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:51.050584 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44a549-db5b-46ca-8e81-6cb44c898f6e service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:51.050862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44a549-db5b-46ca-8e81-6cb44c898f6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:51.051135 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44a549-db5b-46ca-8e81-6cb44c898f6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:51.060309 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa44a549-db5b-46ca-8e81-6cb44c898f6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:51.061758 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa44a549-db5b-46ca-8e81-6cb44c898f6e service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:51.071586 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1252/2540] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:33:50 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:33:51.999805 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6cb43f-89f6-4b5a-a875-ae2155aed40f None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:52.001267 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-1c6cb43f-89f6-4b5a-a875-ae2155aed40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:33:52.013177 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6cb43f-89f6-4b5a-a875-ae2155aed40f None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:52.016759 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae83d6f-d292-4656-9ac8-9dbbd917a535 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:52.017377 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae83d6f-d292-4656-9ac8-9dbbd917a535 service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:52.017657 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae83d6f-d292-4656-9ac8-9dbbd917a535 service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:52.017931 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae83d6f-d292-4656-9ac8-9dbbd917a535 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:52.027797 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:33:52.081398 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae83d6f-d292-4656-9ac8-9dbbd917a535 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:52.082821 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae83d6f-d292-4656-9ac8-9dbbd917a535 service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:52.092468 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1289/2541] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:33:51 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:33:57.877507 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa44a549-db5b-46ca-8e81-6cb44c898f6e service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:33:57.886681 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa44a549-db5b-46ca-8e81-6cb44c898f6e service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:33:57.890756 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdec624-fcdf-40e0-96e0-b38f27738659 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:33:57.891364 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdec624-fcdf-40e0-96e0-b38f27738659 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:33:57.891636 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdec624-fcdf-40e0-96e0-b38f27738659 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:33:57.891902 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdec624-fcdf-40e0-96e0-b38f27738659 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:33:57.932214 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:33:57.961318 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdec624-fcdf-40e0-96e0-b38f27738659 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:33:57.962734 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdec624-fcdf-40e0-96e0-b38f27738659 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:33:57.971809 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1253/2542] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:33:57 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:34:01.484302 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae83d6f-d292-4656-9ac8-9dbbd917a535 service cinder] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:01.547931 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae83d6f-d292-4656-9ac8-9dbbd917a535 service cinder] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:01.552459 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b3bab-9c05-4e9a-b9e0-df1fe2addeac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:01.553087 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b3bab-9c05-4e9a-b9e0-df1fe2addeac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:01.553359 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b3bab-9c05-4e9a-b9e0-df1fe2addeac admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:01.553627 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b3bab-9c05-4e9a-b9e0-df1fe2addeac admin admin] PATH_INFO: `/v3/users/9e2929a96b424fde8f63e763f9610191` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:01.554764 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b3bab-9c05-4e9a-b9e0-df1fe2addeac admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e2929a96b424fde8f63e763f9610191)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:01.556815 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b3bab-9c05-4e9a-b9e0-df1fe2addeac admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:01.737828 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-989b3bab-9c05-4e9a-b9e0-df1fe2addeac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e2929a96b424fde8f63e763f9610191'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:01.744858 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-989b3bab-9c05-4e9a-b9e0-df1fe2addeac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e2929a96b424fde8f63e763f9610191'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:01.745516 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-989b3bab-9c05-4e9a-b9e0-df1fe2addeac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e2929a96b424fde8f63e763f9610191'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:01.746146 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-989b3bab-9c05-4e9a-b9e0-df1fe2addeac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e2929a96b424fde8f63e763f9610191'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:01.757579 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-989b3bab-9c05-4e9a-b9e0-df1fe2addeac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e2929a96b424fde8f63e763f9610191'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:01.827093 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1290/2543] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:34:01 2022] DELETE /identity/v3/users/9e2929a96b424fde8f63e763f9610191 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:34:01.831768 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53cfecd-3ce3-4584-908b-d757e392971f None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:01.832250 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53cfecd-3ce3-4584-908b-d757e392971f None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:01.832449 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53cfecd-3ce3-4584-908b-d757e392971f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:01.851819 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-d53cfecd-3ce3-4584-908b-d757e392971f None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:34:01.853061 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d53cfecd-3ce3-4584-908b-d757e392971f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:01.910297 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1254/2544] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:34:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:34:01.920075 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b3bab-9c05-4e9a-b9e0-df1fe2addeac admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:01.921506 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-989b3bab-9c05-4e9a-b9e0-df1fe2addeac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:01.956616 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:34:01.979638 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b3bab-9c05-4e9a-b9e0-df1fe2addeac admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:01.983734 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff0e46-c259-4f4a-8b11-8a48bd8b20cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:01.984337 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff0e46-c259-4f4a-8b11-8a48bd8b20cd service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:01.984645 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff0e46-c259-4f4a-8b11-8a48bd8b20cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:01.984926 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff0e46-c259-4f4a-8b11-8a48bd8b20cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:01.993853 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ff0e46-c259-4f4a-8b11-8a48bd8b20cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:01.995563 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ff0e46-c259-4f4a-8b11-8a48bd8b20cd service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:02.004979 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1291/2545] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:34:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:34:02.417664 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53cfecd-3ce3-4584-908b-d757e392971f None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:02.419153 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d53cfecd-3ce3-4584-908b-d757e392971f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:02.431010 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53cfecd-3ce3-4584-908b-d757e392971f None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:02.435923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29921165-75f4-44fd-b28a-6932e100bcbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:02.436593 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29921165-75f4-44fd-b28a-6932e100bcbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:02.436872 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29921165-75f4-44fd-b28a-6932e100bcbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:02.437181 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29921165-75f4-44fd-b28a-6932e100bcbc admin admin] PATH_INFO: `/v3/projects/1506f697b12745a9a3dcec792da6d20c` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:02.438402 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29921165-75f4-44fd-b28a-6932e100bcbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=1506f697b12745a9a3dcec792da6d20c)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:02.440300 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29921165-75f4-44fd-b28a-6932e100bcbc admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:02.565870 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-29921165-75f4-44fd-b28a-6932e100bcbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1506f697b12745a9a3dcec792da6d20c', 'request_id': 'req-29921165-75f4-44fd-b28a-6932e100bcbc'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:02.572102 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-29921165-75f4-44fd-b28a-6932e100bcbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1506f697b12745a9a3dcec792da6d20c', 'request_id': 'req-29921165-75f4-44fd-b28a-6932e100bcbc'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:02.633216 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-29921165-75f4-44fd-b28a-6932e100bcbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1506f697b12745a9a3dcec792da6d20c', 'request_id': 'req-29921165-75f4-44fd-b28a-6932e100bcbc'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:02.645536 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-29921165-75f4-44fd-b28a-6932e100bcbc admin admin] The token cache is being invalidate because project 1506f697b12745a9a3dcec792da6d20c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:34:02.645831 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-29921165-75f4-44fd-b28a-6932e100bcbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:02.646469 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-29921165-75f4-44fd-b28a-6932e100bcbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:02.648028 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1255/2546] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:34:02 2022] DELETE /identity/v3/projects/1506f697b12745a9a3dcec792da6d20c => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:34:02.652543 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff0e46-c259-4f4a-8b11-8a48bd8b20cd service neutron] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:02.662207 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff0e46-c259-4f4a-8b11-8a48bd8b20cd service neutron] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:02.717476 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f4c55-0ecb-4fb2-85b9-05f7447fa007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:02.718118 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f4c55-0ecb-4fb2-85b9-05f7447fa007 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:02.718655 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f4c55-0ecb-4fb2-85b9-05f7447fa007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:02.718955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f4c55-0ecb-4fb2-85b9-05f7447fa007 admin admin] PATH_INFO: `/v3/users/d21a2d43eafb44e98f7483ae0c57bf92` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:02.720148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f4c55-0ecb-4fb2-85b9-05f7447fa007 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d21a2d43eafb44e98f7483ae0c57bf92)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:02.722120 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f4c55-0ecb-4fb2-85b9-05f7447fa007 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:02.845623 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-973f4c55-0ecb-4fb2-85b9-05f7447fa007 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd21a2d43eafb44e98f7483ae0c57bf92'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:02.852094 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-973f4c55-0ecb-4fb2-85b9-05f7447fa007 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd21a2d43eafb44e98f7483ae0c57bf92'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:02.852764 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-973f4c55-0ecb-4fb2-85b9-05f7447fa007 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd21a2d43eafb44e98f7483ae0c57bf92'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:02.853397 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-973f4c55-0ecb-4fb2-85b9-05f7447fa007 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd21a2d43eafb44e98f7483ae0c57bf92'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:02.865162 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-973f4c55-0ecb-4fb2-85b9-05f7447fa007 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd21a2d43eafb44e98f7483ae0c57bf92'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:02.946360 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1292/2547] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:34:02 2022] DELETE /identity/v3/users/d21a2d43eafb44e98f7483ae0c57bf92 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:34:03.355420 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29921165-75f4-44fd-b28a-6932e100bcbc admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:03.356934 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-29921165-75f4-44fd-b28a-6932e100bcbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:03.370948 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29921165-75f4-44fd-b28a-6932e100bcbc admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:03.419779 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:34:03.428326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9097e121-e0a7-4444-997a-4b8b18683257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:03.429014 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097e121-e0a7-4444-997a-4b8b18683257 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:03.429289 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097e121-e0a7-4444-997a-4b8b18683257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:03.429560 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097e121-e0a7-4444-997a-4b8b18683257 admin admin] PATH_INFO: `/v3/projects/d8f4d5e128524f1bb0f1905480605d49` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:03.430700 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9097e121-e0a7-4444-997a-4b8b18683257 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8f4d5e128524f1bb0f1905480605d49)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:03.433333 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9097e121-e0a7-4444-997a-4b8b18683257 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:03.525251 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9097e121-e0a7-4444-997a-4b8b18683257 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8f4d5e128524f1bb0f1905480605d49', 'request_id': 'req-9097e121-e0a7-4444-997a-4b8b18683257'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:03.531913 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9097e121-e0a7-4444-997a-4b8b18683257 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8f4d5e128524f1bb0f1905480605d49', 'request_id': 'req-9097e121-e0a7-4444-997a-4b8b18683257'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:03.587096 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9097e121-e0a7-4444-997a-4b8b18683257 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8f4d5e128524f1bb0f1905480605d49', 'request_id': 'req-9097e121-e0a7-4444-997a-4b8b18683257'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:03.598011 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9097e121-e0a7-4444-997a-4b8b18683257 admin admin] The token cache is being invalidate because project d8f4d5e128524f1bb0f1905480605d49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:34:03.598290 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9097e121-e0a7-4444-997a-4b8b18683257 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:03.598897 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9097e121-e0a7-4444-997a-4b8b18683257 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:03.600384 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1256/2548] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:34:03 2022] DELETE /identity/v3/projects/d8f4d5e128524f1bb0f1905480605d49 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:34:03.604873 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f4c55-0ecb-4fb2-85b9-05f7447fa007 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:03.606517 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-973f4c55-0ecb-4fb2-85b9-05f7447fa007 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:03.620657 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f4c55-0ecb-4fb2-85b9-05f7447fa007 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:03.665434 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:34:03.674606 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69aeea7-6ae7-4a0d-b122-a92d8ddc23bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:03.677813 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69aeea7-6ae7-4a0d-b122-a92d8ddc23bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:03.677813 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69aeea7-6ae7-4a0d-b122-a92d8ddc23bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:03.677813 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69aeea7-6ae7-4a0d-b122-a92d8ddc23bb admin admin] PATH_INFO: `/v3/users/ca6c80c6e5264437a62e7667a0279f14` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:03.677813 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69aeea7-6ae7-4a0d-b122-a92d8ddc23bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca6c80c6e5264437a62e7667a0279f14)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:03.678875 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69aeea7-6ae7-4a0d-b122-a92d8ddc23bb admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:03.847061 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-c69aeea7-6ae7-4a0d-b122-a92d8ddc23bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca6c80c6e5264437a62e7667a0279f14'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:03.853903 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-c69aeea7-6ae7-4a0d-b122-a92d8ddc23bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca6c80c6e5264437a62e7667a0279f14'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:03.855162 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-c69aeea7-6ae7-4a0d-b122-a92d8ddc23bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca6c80c6e5264437a62e7667a0279f14'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:03.855772 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-c69aeea7-6ae7-4a0d-b122-a92d8ddc23bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca6c80c6e5264437a62e7667a0279f14'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:03.867528 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-c69aeea7-6ae7-4a0d-b122-a92d8ddc23bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca6c80c6e5264437a62e7667a0279f14'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:03.930178 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1293/2549] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:34:03 2022] DELETE /identity/v3/users/ca6c80c6e5264437a62e7667a0279f14 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:34:04.349192 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9097e121-e0a7-4444-997a-4b8b18683257 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:04.350606 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-9097e121-e0a7-4444-997a-4b8b18683257 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:04.363574 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9097e121-e0a7-4444-997a-4b8b18683257 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:04.422076 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2e0059-efc2-4df2-8bf0-61fbcadca432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:04.422718 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e0059-efc2-4df2-8bf0-61fbcadca432 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:04.422995 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e0059-efc2-4df2-8bf0-61fbcadca432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:04.423267 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e0059-efc2-4df2-8bf0-61fbcadca432 admin admin] PATH_INFO: `/v3/projects/9d5e81234b9d4e71af079fb3049c041f` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:04.424441 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2e0059-efc2-4df2-8bf0-61fbcadca432 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d5e81234b9d4e71af079fb3049c041f)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:04.426436 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2e0059-efc2-4df2-8bf0-61fbcadca432 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:04.535759 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-3b2e0059-efc2-4df2-8bf0-61fbcadca432 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d5e81234b9d4e71af079fb3049c041f', 'request_id': 'req-3b2e0059-efc2-4df2-8bf0-61fbcadca432'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:04.542084 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-3b2e0059-efc2-4df2-8bf0-61fbcadca432 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d5e81234b9d4e71af079fb3049c041f', 'request_id': 'req-3b2e0059-efc2-4df2-8bf0-61fbcadca432'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:04.597555 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-3b2e0059-efc2-4df2-8bf0-61fbcadca432 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d5e81234b9d4e71af079fb3049c041f', 'request_id': 'req-3b2e0059-efc2-4df2-8bf0-61fbcadca432'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:04.608140 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-3b2e0059-efc2-4df2-8bf0-61fbcadca432 admin admin] The token cache is being invalidate because project 9d5e81234b9d4e71af079fb3049c041f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:34:04.608455 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-3b2e0059-efc2-4df2-8bf0-61fbcadca432 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:04.609128 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-3b2e0059-efc2-4df2-8bf0-61fbcadca432 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:04.610629 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1257/2550] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:34:04 2022] DELETE /identity/v3/projects/9d5e81234b9d4e71af079fb3049c041f => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:34:04.616668 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b913c-071e-41e2-9360-e6f7668e0904 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:04.616952 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b913c-071e-41e2-9360-e6f7668e0904 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:04.617226 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b913c-071e-41e2-9360-e6f7668e0904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:04.638822 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-852b913c-071e-41e2-9360-e6f7668e0904 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:34:04.640011 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-852b913c-071e-41e2-9360-e6f7668e0904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:04.701500 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1294/2551] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:34:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:34:04.773564 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83083f0e-98ca-4161-9cef-6f716a7c7425 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:04.773783 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83083f0e-98ca-4161-9cef-6f716a7c7425 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:04.774024 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83083f0e-98ca-4161-9cef-6f716a7c7425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:04.795903 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-83083f0e-98ca-4161-9cef-6f716a7c7425 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:34:04.797172 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-83083f0e-98ca-4161-9cef-6f716a7c7425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:04.806006 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1258/2552] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:34:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:34:04.810669 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852b913c-071e-41e2-9360-e6f7668e0904 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:04.820827 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852b913c-071e-41e2-9360-e6f7668e0904 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:04.825928 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102d5df-462f-4462-8359-f1ac977cb92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:04.826545 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102d5df-462f-4462-8359-f1ac977cb92b admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:04.826812 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102d5df-462f-4462-8359-f1ac977cb92b admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:04.827075 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102d5df-462f-4462-8359-f1ac977cb92b admin admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:04.827798 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5102d5df-462f-4462-8359-f1ac977cb92b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:34:04.828093 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5102d5df-462f-4462-8359-f1ac977cb92b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:04.829709 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5102d5df-462f-4462-8359-f1ac977cb92b admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:04.842938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1295/2553] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:34:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:34:04.847281 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83083f0e-98ca-4161-9cef-6f716a7c7425 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:04.857784 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83083f0e-98ca-4161-9cef-6f716a7c7425 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:04.864093 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbca9eb-5a82-4a2e-bef9-e78342e89d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:04.864743 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbca9eb-5a82-4a2e-bef9-e78342e89d09 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:04.865019 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbca9eb-5a82-4a2e-bef9-e78342e89d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:04.865289 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbca9eb-5a82-4a2e-bef9-e78342e89d09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:04.865946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbca9eb-5a82-4a2e-bef9-e78342e89d09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:04.867907 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbca9eb-5a82-4a2e-bef9-e78342e89d09 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:04.872903 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:34:04.946230 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1259/2554] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:34:04 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:34:04.951288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102d5df-462f-4462-8359-f1ac977cb92b admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:04.965149 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102d5df-462f-4462-8359-f1ac977cb92b admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:05.032139 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0b2556-aea2-4754-991d-c2d5afdff3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:05.032720 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b2556-aea2-4754-991d-c2d5afdff3f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:05.033090 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b2556-aea2-4754-991d-c2d5afdff3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:05.033360 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b2556-aea2-4754-991d-c2d5afdff3f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:05.034013 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0b2556-aea2-4754-991d-c2d5afdff3f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:05.035895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0b2556-aea2-4754-991d-c2d5afdff3f7 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:05.044102 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:34:05.102340 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1296/2555] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:34:04 2022] POST /identity/v3/users => generated 290 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:34:05.106623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbca9eb-5a82-4a2e-bef9-e78342e89d09 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:05.117494 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbca9eb-5a82-4a2e-bef9-e78342e89d09 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:05.122335 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8ba1c-e9dc-477b-a38d-8eabbf8601c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:05.122952 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8ba1c-e9dc-477b-a38d-8eabbf8601c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:05.123223 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8ba1c-e9dc-477b-a38d-8eabbf8601c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:05.123489 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8ba1c-e9dc-477b-a38d-8eabbf8601c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:05.124174 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a8ba1c-e9dc-477b-a38d-8eabbf8601c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:05.125799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a8ba1c-e9dc-477b-a38d-8eabbf8601c3 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:05.137089 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1260/2556] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:34:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:34:05.140889 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0b2556-aea2-4754-991d-c2d5afdff3f7 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:05.151942 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0b2556-aea2-4754-991d-c2d5afdff3f7 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:05.157407 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1567dcf-4d77-4388-bd72-27537edaffa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:05.158120 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1567dcf-4d77-4388-bd72-27537edaffa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:05.158460 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1567dcf-4d77-4388-bd72-27537edaffa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:05.158799 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1567dcf-4d77-4388-bd72-27537edaffa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:05.159551 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1567dcf-4d77-4388-bd72-27537edaffa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:05.161245 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1567dcf-4d77-4388-bd72-27537edaffa5 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:05.173466 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1297/2557] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:34:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:34:05.177551 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8ba1c-e9dc-477b-a38d-8eabbf8601c3 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:05.187765 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8ba1c-e9dc-477b-a38d-8eabbf8601c3 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:05.192519 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc76549-a57e-4353-9cb4-f1ec5ac1402d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:05.193235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc76549-a57e-4353-9cb4-f1ec5ac1402d admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:05.193508 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc76549-a57e-4353-9cb4-f1ec5ac1402d admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:05.193779 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc76549-a57e-4353-9cb4-f1ec5ac1402d admin admin] PATH_INFO: `/v3/projects/d79be04d47d54344a053527e2177f840/users/34b0776ce69246ebb8ad1a628819c654/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:05.209639 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc76549-a57e-4353-9cb4-f1ec5ac1402d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d79be04d47d54344a053527e2177f840, user_id=34b0776ce69246ebb8ad1a628819c654, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:05.212317 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc76549-a57e-4353-9cb4-f1ec5ac1402d admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:05.288720 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1261/2558] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:34:05 2022] PUT /identity/v3/projects/d79be04d47d54344a053527e2177f840/users/34b0776ce69246ebb8ad1a628819c654/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:34:05.300187 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660443e0-d959-41ac-b0bd-50a051602497 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:05.300484 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660443e0-d959-41ac-b0bd-50a051602497 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:05.300752 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660443e0-d959-41ac-b0bd-50a051602497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:05.328321 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-660443e0-d959-41ac-b0bd-50a051602497 None None] MFA Rules not processed for user `34b0776ce69246ebb8ad1a628819c654`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:34:05.329565 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-660443e0-d959-41ac-b0bd-50a051602497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:05.392008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1298/2559] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:34:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:34:05.396689 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc76549-a57e-4353-9cb4-f1ec5ac1402d admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:05.456555 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc76549-a57e-4353-9cb4-f1ec5ac1402d admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:05.461274 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244b597b-c76e-4366-ad84-81a07a11259e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:05.461882 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b597b-c76e-4366-ad84-81a07a11259e admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:05.462161 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b597b-c76e-4366-ad84-81a07a11259e admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:05.462436 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b597b-c76e-4366-ad84-81a07a11259e admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:05.463098 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244b597b-c76e-4366-ad84-81a07a11259e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:05.465562 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244b597b-c76e-4366-ad84-81a07a11259e admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:05.549366 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1262/2560] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:34:05 2022] POST /identity/v3/projects => generated 381 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:34:05.553669 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660443e0-d959-41ac-b0bd-50a051602497 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:05.564203 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660443e0-d959-41ac-b0bd-50a051602497 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:05.618329 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91889d4a-0143-4534-adc9-238be8a1966e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:05.618944 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91889d4a-0143-4534-adc9-238be8a1966e admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:05.619225 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91889d4a-0143-4534-adc9-238be8a1966e admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:05.619500 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91889d4a-0143-4534-adc9-238be8a1966e admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:05.620497 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91889d4a-0143-4534-adc9-238be8a1966e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:05.622369 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91889d4a-0143-4534-adc9-238be8a1966e admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:05.736494 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1299/2561] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:34:05 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:34:05.740268 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244b597b-c76e-4366-ad84-81a07a11259e admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:05.750505 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244b597b-c76e-4366-ad84-81a07a11259e admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:05.755170 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8941b5-aa49-43d8-8b36-d315fbdfb92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:05.755738 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8941b5-aa49-43d8-8b36-d315fbdfb92c admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:05.756019 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8941b5-aa49-43d8-8b36-d315fbdfb92c admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:05.756290 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8941b5-aa49-43d8-8b36-d315fbdfb92c admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:05.757012 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8941b5-aa49-43d8-8b36-d315fbdfb92c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:05.758603 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8941b5-aa49-43d8-8b36-d315fbdfb92c admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:05.769568 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1263/2562] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:34:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:34:05.773738 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91889d4a-0143-4534-adc9-238be8a1966e admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:05.784291 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91889d4a-0143-4534-adc9-238be8a1966e admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:05.789082 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e177b-f04a-46c4-929f-dda37ec52a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:05.789745 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e177b-f04a-46c4-929f-dda37ec52a9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:05.790026 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e177b-f04a-46c4-929f-dda37ec52a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:05.790299 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e177b-f04a-46c4-929f-dda37ec52a9a admin admin] PATH_INFO: `/v3/projects/5447e6bf5d2e4fe4aa7ab8d4af9b58b6/users/30b110b19dc24102b6f9b357b36de4f9/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:05.804937 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e177b-f04a-46c4-929f-dda37ec52a9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5447e6bf5d2e4fe4aa7ab8d4af9b58b6, user_id=30b110b19dc24102b6f9b357b36de4f9, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:05.807531 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e177b-f04a-46c4-929f-dda37ec52a9a admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:05.867468 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1300/2563] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:34:05 2022] PUT /identity/v3/projects/5447e6bf5d2e4fe4aa7ab8d4af9b58b6/users/30b110b19dc24102b6f9b357b36de4f9/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:34:05.871634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8941b5-aa49-43d8-8b36-d315fbdfb92c admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:05.881739 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8941b5-aa49-43d8-8b36-d315fbdfb92c admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:05.938841 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87237c7-b5ba-4603-bebb-21d398b4fd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:05.939453 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87237c7-b5ba-4603-bebb-21d398b4fd2e admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:05.939736 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87237c7-b5ba-4603-bebb-21d398b4fd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:05.940005 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87237c7-b5ba-4603-bebb-21d398b4fd2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:05.940881 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87237c7-b5ba-4603-bebb-21d398b4fd2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:05.945821 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87237c7-b5ba-4603-bebb-21d398b4fd2e admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:05.953319 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1264/2564] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:34:05 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:34:05.957496 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e177b-f04a-46c4-929f-dda37ec52a9a admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:05.968672 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e177b-f04a-46c4-929f-dda37ec52a9a admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:05.973297 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbeeac3-c59c-4fb9-8afe-d76d16b57db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:05.973948 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbeeac3-c59c-4fb9-8afe-d76d16b57db5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:05.974222 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbeeac3-c59c-4fb9-8afe-d76d16b57db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:05.974487 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbeeac3-c59c-4fb9-8afe-d76d16b57db5 admin admin] PATH_INFO: `/v3/projects/5447e6bf5d2e4fe4aa7ab8d4af9b58b6/users/30b110b19dc24102b6f9b357b36de4f9/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:05.976645 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbeeac3-c59c-4fb9-8afe-d76d16b57db5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5447e6bf5d2e4fe4aa7ab8d4af9b58b6, user_id=30b110b19dc24102b6f9b357b36de4f9, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:05.979378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbeeac3-c59c-4fb9-8afe-d76d16b57db5 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:05.990215 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1301/2565] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:34:05 2022] PUT /identity/v3/projects/5447e6bf5d2e4fe4aa7ab8d4af9b58b6/users/30b110b19dc24102b6f9b357b36de4f9/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:34:06.006069 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4478d68-2952-4be6-80a0-bff29452b645 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:06.006355 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4478d68-2952-4be6-80a0-bff29452b645 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:06.006628 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4478d68-2952-4be6-80a0-bff29452b645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:06.035249 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-b4478d68-2952-4be6-80a0-bff29452b645 None None] MFA Rules not processed for user `30b110b19dc24102b6f9b357b36de4f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:34:06.036485 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-b4478d68-2952-4be6-80a0-bff29452b645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:06.045496 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:34:06.106874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1265/2566] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:34:06 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:34:06.117584 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbeeac3-c59c-4fb9-8afe-d76d16b57db5 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:06.119394 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-fcbeeac3-c59c-4fb9-8afe-d76d16b57db5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:06.143677 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:34:06.180202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbeeac3-c59c-4fb9-8afe-d76d16b57db5 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:06.184748 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332703c2-b901-49b6-a5be-8f6a9f67291d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:06.185377 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332703c2-b901-49b6-a5be-8f6a9f67291d service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:06.185658 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332703c2-b901-49b6-a5be-8f6a9f67291d service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:06.185929 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332703c2-b901-49b6-a5be-8f6a9f67291d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:06.195954 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332703c2-b901-49b6-a5be-8f6a9f67291d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:06.197640 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332703c2-b901-49b6-a5be-8f6a9f67291d service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:06.209008 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1302/2567] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:34:06 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:34:06.795568 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4478d68-2952-4be6-80a0-bff29452b645 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:06.870205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4478d68-2952-4be6-80a0-bff29452b645 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:06.874819 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb9bfd-162a-42b6-b5c6-a154e2b263e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:06.875459 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb9bfd-162a-42b6-b5c6-a154e2b263e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:06.875738 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb9bfd-162a-42b6-b5c6-a154e2b263e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:06.876033 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb9bfd-162a-42b6-b5c6-a154e2b263e2 admin admin] PATH_INFO: `/v3/users/34b0776ce69246ebb8ad1a628819c654` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:06.877245 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fb9bfd-162a-42b6-b5c6-a154e2b263e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34b0776ce69246ebb8ad1a628819c654)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:06.879171 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fb9bfd-162a-42b6-b5c6-a154e2b263e2 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:07.091713 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c0fb9bfd-162a-42b6-b5c6-a154e2b263e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34b0776ce69246ebb8ad1a628819c654'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:07.103596 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c0fb9bfd-162a-42b6-b5c6-a154e2b263e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34b0776ce69246ebb8ad1a628819c654'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:07.204663 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c0fb9bfd-162a-42b6-b5c6-a154e2b263e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34b0776ce69246ebb8ad1a628819c654'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:07.205241 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c0fb9bfd-162a-42b6-b5c6-a154e2b263e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34b0776ce69246ebb8ad1a628819c654'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:07.212806 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c0fb9bfd-162a-42b6-b5c6-a154e2b263e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34b0776ce69246ebb8ad1a628819c654'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:07.220109 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1266/2568] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:34:06 2022] DELETE /identity/v3/users/34b0776ce69246ebb8ad1a628819c654 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:34:07.224789 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb058a-dd29-4c8a-81d2-5b1d5abb36b2 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:07.225242 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb058a-dd29-4c8a-81d2-5b1d5abb36b2 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:07.225343 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb058a-dd29-4c8a-81d2-5b1d5abb36b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:07.244849 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-10bb058a-dd29-4c8a-81d2-5b1d5abb36b2 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:34:07.246420 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-10bb058a-dd29-4c8a-81d2-5b1d5abb36b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:07.306788 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1303/2569] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:34:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:34:07.316545 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb9bfd-162a-42b6-b5c6-a154e2b263e2 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:07.318019 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c0fb9bfd-162a-42b6-b5c6-a154e2b263e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:07.347294 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:34:07.378022 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb9bfd-162a-42b6-b5c6-a154e2b263e2 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:07.381571 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e16e1-5fcc-4274-b95f-d65feea651ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:07.382182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e16e1-5fcc-4274-b95f-d65feea651ab service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:07.382453 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e16e1-5fcc-4274-b95f-d65feea651ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:07.382721 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e16e1-5fcc-4274-b95f-d65feea651ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:07.392954 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113e16e1-5fcc-4274-b95f-d65feea651ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:07.394370 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113e16e1-5fcc-4274-b95f-d65feea651ab service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:07.404325 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1267/2570] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:34:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:34:07.799729 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb058a-dd29-4c8a-81d2-5b1d5abb36b2 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:07.801198 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-10bb058a-dd29-4c8a-81d2-5b1d5abb36b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:07.813459 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb058a-dd29-4c8a-81d2-5b1d5abb36b2 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:07.818131 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a5829-9e80-4ed9-bdec-0a012f47a9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:07.818771 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a5829-9e80-4ed9-bdec-0a012f47a9b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:07.819053 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a5829-9e80-4ed9-bdec-0a012f47a9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:07.819327 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a5829-9e80-4ed9-bdec-0a012f47a9b0 admin admin] PATH_INFO: `/v3/projects/d79be04d47d54344a053527e2177f840` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:07.820546 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a5829-9e80-4ed9-bdec-0a012f47a9b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d79be04d47d54344a053527e2177f840)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:07.822449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a5829-9e80-4ed9-bdec-0a012f47a9b0 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:07.911251 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-bc7a5829-9e80-4ed9-bdec-0a012f47a9b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd79be04d47d54344a053527e2177f840', 'request_id': 'req-bc7a5829-9e80-4ed9-bdec-0a012f47a9b0'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:07.917340 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-bc7a5829-9e80-4ed9-bdec-0a012f47a9b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd79be04d47d54344a053527e2177f840', 'request_id': 'req-bc7a5829-9e80-4ed9-bdec-0a012f47a9b0'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:07.931436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-bc7a5829-9e80-4ed9-bdec-0a012f47a9b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd79be04d47d54344a053527e2177f840', 'request_id': 'req-bc7a5829-9e80-4ed9-bdec-0a012f47a9b0'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:08.002996 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-bc7a5829-9e80-4ed9-bdec-0a012f47a9b0 admin admin] The token cache is being invalidate because project d79be04d47d54344a053527e2177f840 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:34:08.003277 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-bc7a5829-9e80-4ed9-bdec-0a012f47a9b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:08.003904 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-bc7a5829-9e80-4ed9-bdec-0a012f47a9b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:08.005492 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1304/2571] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:34:07 2022] DELETE /identity/v3/projects/d79be04d47d54344a053527e2177f840 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:34:08.009449 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e16e1-5fcc-4274-b95f-d65feea651ab service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:08.019932 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e16e1-5fcc-4274-b95f-d65feea651ab service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:08.074843 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e656ef-eae7-40dc-bc53-f087f6969e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:08.075373 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e656ef-eae7-40dc-bc53-f087f6969e28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:08.075650 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e656ef-eae7-40dc-bc53-f087f6969e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:08.075922 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e656ef-eae7-40dc-bc53-f087f6969e28 admin admin] PATH_INFO: `/v3/users/30b110b19dc24102b6f9b357b36de4f9` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:08.077108 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e656ef-eae7-40dc-bc53-f087f6969e28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30b110b19dc24102b6f9b357b36de4f9)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:08.079011 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e656ef-eae7-40dc-bc53-f087f6969e28 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:08.293050 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f0e656ef-eae7-40dc-bc53-f087f6969e28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30b110b19dc24102b6f9b357b36de4f9'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:08.304615 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f0e656ef-eae7-40dc-bc53-f087f6969e28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30b110b19dc24102b6f9b357b36de4f9'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:08.393274 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f0e656ef-eae7-40dc-bc53-f087f6969e28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30b110b19dc24102b6f9b357b36de4f9'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:08.394914 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f0e656ef-eae7-40dc-bc53-f087f6969e28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30b110b19dc24102b6f9b357b36de4f9'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:08.403186 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f0e656ef-eae7-40dc-bc53-f087f6969e28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30b110b19dc24102b6f9b357b36de4f9'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:08.410030 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1268/2572] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:34:08 2022] DELETE /identity/v3/users/30b110b19dc24102b6f9b357b36de4f9 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:34:08.795692 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a30f0-5052-4f01-80fd-1f24d60a040f None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:08.797305 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a30f0-5052-4f01-80fd-1f24d60a040f None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:08.797305 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a30f0-5052-4f01-80fd-1f24d60a040f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:08.826413 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-901a30f0-5052-4f01-80fd-1f24d60a040f None None] MFA Rules not processed for user `17e3b509a2de4159bf234594573f5e18`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:34:08.827618 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-901a30f0-5052-4f01-80fd-1f24d60a040f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:08.861803 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:34:08.892052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1305/2573] 10.222.0.26 () {58 vars in 1009 bytes} [Fri Aug 5 14:34:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:34:08.904681 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e656ef-eae7-40dc-bc53-f087f6969e28 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:08.906205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f0e656ef-eae7-40dc-bc53-f087f6969e28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:08.965460 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:34:08.981849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e656ef-eae7-40dc-bc53-f087f6969e28 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:08.986560 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7607e5-8560-4596-b31c-4ef01d7d0a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:08.987107 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7607e5-8560-4596-b31c-4ef01d7d0a64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:08.987383 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7607e5-8560-4596-b31c-4ef01d7d0a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:08.987653 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7607e5-8560-4596-b31c-4ef01d7d0a64 admin admin] PATH_INFO: `/v3/projects/5447e6bf5d2e4fe4aa7ab8d4af9b58b6` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:08.988833 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7607e5-8560-4596-b31c-4ef01d7d0a64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5447e6bf5d2e4fe4aa7ab8d4af9b58b6)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:08.990731 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7607e5-8560-4596-b31c-4ef01d7d0a64 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:09.021416 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a30f0-5052-4f01-80fd-1f24d60a040f None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:09.022881 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-901a30f0-5052-4f01-80fd-1f24d60a040f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:09.094354 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a30f0-5052-4f01-80fd-1f24d60a040f None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:09.099483 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d622cc-c747-409e-b6d2-6b8f1c17af2f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dddd7eea67f34f0cbb5f6a1cd85f1fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:09.100097 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d622cc-c747-409e-b6d2-6b8f1c17af2f service swift] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:09.100378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d622cc-c747-409e-b6d2-6b8f1c17af2f service swift] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:09.100683 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d622cc-c747-409e-b6d2-6b8f1c17af2f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:09.109568 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d622cc-c747-409e-b6d2-6b8f1c17af2f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:09.111203 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d622cc-c747-409e-b6d2-6b8f1c17af2f service swift] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:09.119612 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1306/2574] 10.222.0.26 () {60 vars in 1384 bytes} [Fri Aug 5 14:34:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 05 14:34:09.132967 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ca7607e5-8560-4596-b31c-4ef01d7d0a64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5447e6bf5d2e4fe4aa7ab8d4af9b58b6', 'request_id': 'req-ca7607e5-8560-4596-b31c-4ef01d7d0a64'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:09.139900 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ca7607e5-8560-4596-b31c-4ef01d7d0a64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5447e6bf5d2e4fe4aa7ab8d4af9b58b6', 'request_id': 'req-ca7607e5-8560-4596-b31c-4ef01d7d0a64'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:09.215155 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ca7607e5-8560-4596-b31c-4ef01d7d0a64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5447e6bf5d2e4fe4aa7ab8d4af9b58b6', 'request_id': 'req-ca7607e5-8560-4596-b31c-4ef01d7d0a64'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:09.228037 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ca7607e5-8560-4596-b31c-4ef01d7d0a64 admin admin] The token cache is being invalidate because project 5447e6bf5d2e4fe4aa7ab8d4af9b58b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:34:09.228939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ca7607e5-8560-4596-b31c-4ef01d7d0a64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:09.229939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ca7607e5-8560-4596-b31c-4ef01d7d0a64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:09.232239 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1269/2575] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:34:08 2022] DELETE /identity/v3/projects/5447e6bf5d2e4fe4aa7ab8d4af9b58b6 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:34:09.237162 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f8f6a-f35e-44f8-957b-49cfee986b61 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:09.237519 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f8f6a-f35e-44f8-957b-49cfee986b61 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:09.237965 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f8f6a-f35e-44f8-957b-49cfee986b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:09.261263 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-db9f8f6a-f35e-44f8-957b-49cfee986b61 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:34:09.262501 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-db9f8f6a-f35e-44f8-957b-49cfee986b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:09.331918 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1307/2576] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:34:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:34:09.341057 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfb23a-5df6-4a47-864c-50d17425cdf0 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:09.341572 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfb23a-5df6-4a47-864c-50d17425cdf0 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:09.342029 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfb23a-5df6-4a47-864c-50d17425cdf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:09.363341 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-3edfb23a-5df6-4a47-864c-50d17425cdf0 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:34:09.364599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-3edfb23a-5df6-4a47-864c-50d17425cdf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:09.374929 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1270/2577] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:34:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:34:09.379040 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f8f6a-f35e-44f8-957b-49cfee986b61 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:09.390528 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f8f6a-f35e-44f8-957b-49cfee986b61 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:09.396317 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d2bf7c-38bc-43e0-9c99-30cfbe93d2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:09.396823 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2bf7c-38bc-43e0-9c99-30cfbe93d2a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:09.397151 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2bf7c-38bc-43e0-9c99-30cfbe93d2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:09.397425 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2bf7c-38bc-43e0-9c99-30cfbe93d2a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:09.398114 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d2bf7c-38bc-43e0-9c99-30cfbe93d2a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:34:09.398413 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d2bf7c-38bc-43e0-9c99-30cfbe93d2a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:09.399991 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d2bf7c-38bc-43e0-9c99-30cfbe93d2a2 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:09.416934 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1308/2578] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:34:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:34:09.418300 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edfb23a-5df6-4a47-864c-50d17425cdf0 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:09.427955 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edfb23a-5df6-4a47-864c-50d17425cdf0 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:09.432883 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7b9f9-9ee1-41cf-b262-89707e1848cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:09.433487 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7b9f9-9ee1-41cf-b262-89707e1848cb admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:09.433769 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7b9f9-9ee1-41cf-b262-89707e1848cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:09.434040 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7b9f9-9ee1-41cf-b262-89707e1848cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:09.434725 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b7b9f9-9ee1-41cf-b262-89707e1848cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:09.436652 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b7b9f9-9ee1-41cf-b262-89707e1848cb admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:09.543150 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1271/2579] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:34:09 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:34:09.547195 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d2bf7c-38bc-43e0-9c99-30cfbe93d2a2 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:09.559586 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d2bf7c-38bc-43e0-9c99-30cfbe93d2a2 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:09.617873 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:34:09.631414 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ebfa29-602e-4bd6-bc5f-05da0d5c1e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:09.632501 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ebfa29-602e-4bd6-bc5f-05da0d5c1e2d admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:09.632501 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ebfa29-602e-4bd6-bc5f-05da0d5c1e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:09.632664 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ebfa29-602e-4bd6-bc5f-05da0d5c1e2d admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:09.633446 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ebfa29-602e-4bd6-bc5f-05da0d5c1e2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:09.635371 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ebfa29-602e-4bd6-bc5f-05da0d5c1e2d admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:09.721098 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1309/2580] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:34:09 2022] POST /identity/v3/users => generated 293 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:34:09.725189 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7b9f9-9ee1-41cf-b262-89707e1848cb admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:09.736715 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7b9f9-9ee1-41cf-b262-89707e1848cb admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:09.743209 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10ea85-847b-4b01-998f-c1050e8cc139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:09.743843 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10ea85-847b-4b01-998f-c1050e8cc139 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:09.744570 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10ea85-847b-4b01-998f-c1050e8cc139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:09.744570 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10ea85-847b-4b01-998f-c1050e8cc139 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:09.745955 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10ea85-847b-4b01-998f-c1050e8cc139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:09.747550 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10ea85-847b-4b01-998f-c1050e8cc139 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:09.763904 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1272/2581] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:34:09.764998 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ebfa29-602e-4bd6-bc5f-05da0d5c1e2d admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:09.777925 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ebfa29-602e-4bd6-bc5f-05da0d5c1e2d admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:09.783734 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd1c289-3dfd-433a-9946-3cf450f4305b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:09.786121 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1c289-3dfd-433a-9946-3cf450f4305b admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:09.786788 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1c289-3dfd-433a-9946-3cf450f4305b admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:09.787048 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1c289-3dfd-433a-9946-3cf450f4305b admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:09.787771 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd1c289-3dfd-433a-9946-3cf450f4305b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:09.789660 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd1c289-3dfd-433a-9946-3cf450f4305b admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:09.806263 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1310/2582] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:34:09.806954 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10ea85-847b-4b01-998f-c1050e8cc139 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:09.823695 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10ea85-847b-4b01-998f-c1050e8cc139 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:09.830653 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0d27c1-6129-4fd9-b878-0d9d175ddeba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:09.834641 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d27c1-6129-4fd9-b878-0d9d175ddeba admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:09.834641 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d27c1-6129-4fd9-b878-0d9d175ddeba admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:09.834641 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d27c1-6129-4fd9-b878-0d9d175ddeba admin admin] PATH_INFO: `/v3/projects/d7bf9ba57c4c442a90180ed57eded8e9/users/33df75bf5b6a4115af55912415740181/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:09.870881 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0d27c1-6129-4fd9-b878-0d9d175ddeba admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7bf9ba57c4c442a90180ed57eded8e9, user_id=33df75bf5b6a4115af55912415740181, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:09.873772 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0d27c1-6129-4fd9-b878-0d9d175ddeba admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:09.955658 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1273/2583] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:34:09 2022] PUT /identity/v3/projects/d7bf9ba57c4c442a90180ed57eded8e9/users/33df75bf5b6a4115af55912415740181/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:34:09.966812 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4396922e-163c-401a-90e0-27c21e413ee8 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:09.967092 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4396922e-163c-401a-90e0-27c21e413ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:09.967370 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4396922e-163c-401a-90e0-27c21e413ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:10.002419 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-4396922e-163c-401a-90e0-27c21e413ee8 None None] MFA Rules not processed for user `33df75bf5b6a4115af55912415740181`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:34:10.003721 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-4396922e-163c-401a-90e0-27c21e413ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:10.077427 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1311/2584] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:34:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:34:10.090514 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0d27c1-6129-4fd9-b878-0d9d175ddeba admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:10.092361 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-da0d27c1-6129-4fd9-b878-0d9d175ddeba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:10.171535 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0d27c1-6129-4fd9-b878-0d9d175ddeba admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:10.175504 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e839ab4d-1471-403f-839c-3d4cc13e78b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:10.176199 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839ab4d-1471-403f-839c-3d4cc13e78b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:10.176712 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839ab4d-1471-403f-839c-3d4cc13e78b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:10.177059 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839ab4d-1471-403f-839c-3d4cc13e78b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:10.187059 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e839ab4d-1471-403f-839c-3d4cc13e78b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:10.188750 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e839ab4d-1471-403f-839c-3d4cc13e78b7 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:10.199818 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1274/2585] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:34:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:34:13.305747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4396922e-163c-401a-90e0-27c21e413ee8 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:13.411124 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4396922e-163c-401a-90e0-27c21e413ee8 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:13.417728 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e780811a-0e84-490f-af64-668a7913426f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:13.418391 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780811a-0e84-490f-af64-668a7913426f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:13.418667 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780811a-0e84-490f-af64-668a7913426f admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:13.418933 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780811a-0e84-490f-af64-668a7913426f admin admin] PATH_INFO: `/v3/users/33df75bf5b6a4115af55912415740181` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:13.420141 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e780811a-0e84-490f-af64-668a7913426f admin admin] RBAC: Authorizing `identity:delete_user(user_id=33df75bf5b6a4115af55912415740181)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:13.422126 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e780811a-0e84-490f-af64-668a7913426f admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:13.624891 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-e780811a-0e84-490f-af64-668a7913426f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33df75bf5b6a4115af55912415740181'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:13.632028 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-e780811a-0e84-490f-af64-668a7913426f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33df75bf5b6a4115af55912415740181'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:13.632964 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-e780811a-0e84-490f-af64-668a7913426f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33df75bf5b6a4115af55912415740181'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:13.633636 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-e780811a-0e84-490f-af64-668a7913426f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33df75bf5b6a4115af55912415740181'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:13.639838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:34:13.648035 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-e780811a-0e84-490f-af64-668a7913426f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33df75bf5b6a4115af55912415740181'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:13.720810 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1312/2586] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:34:13 2022] DELETE /identity/v3/users/33df75bf5b6a4115af55912415740181 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:34:13.725452 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d044c79-9d00-460e-b4d8-8c0d29b37a9a None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:13.725971 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d044c79-9d00-460e-b4d8-8c0d29b37a9a None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:13.725971 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d044c79-9d00-460e-b4d8-8c0d29b37a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:13.746827 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-5d044c79-9d00-460e-b4d8-8c0d29b37a9a None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:34:13.748094 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5d044c79-9d00-460e-b4d8-8c0d29b37a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:13.759082 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:34:13.815699 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1275/2587] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:34:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:34:13.826061 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e780811a-0e84-490f-af64-668a7913426f admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:13.829781 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-e780811a-0e84-490f-af64-668a7913426f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:13.892771 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e780811a-0e84-490f-af64-668a7913426f admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:13.896993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db65157b-2ebc-4304-a859-062382820034 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:13.897616 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65157b-2ebc-4304-a859-062382820034 service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:13.897902 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65157b-2ebc-4304-a859-062382820034 service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:13.898177 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65157b-2ebc-4304-a859-062382820034 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:13.908328 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db65157b-2ebc-4304-a859-062382820034 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:13.909829 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db65157b-2ebc-4304-a859-062382820034 service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:13.919949 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1313/2588] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:34:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:34:14.384293 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d044c79-9d00-460e-b4d8-8c0d29b37a9a None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:14.385903 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5d044c79-9d00-460e-b4d8-8c0d29b37a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:14.426545 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d044c79-9d00-460e-b4d8-8c0d29b37a9a None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:14.433358 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221fb725-39ae-4295-a9bf-32ad473da9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:14.434366 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221fb725-39ae-4295-a9bf-32ad473da9ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:14.434815 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221fb725-39ae-4295-a9bf-32ad473da9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:14.435270 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221fb725-39ae-4295-a9bf-32ad473da9ba admin admin] PATH_INFO: `/v3/projects/d7bf9ba57c4c442a90180ed57eded8e9` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:14.437152 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221fb725-39ae-4295-a9bf-32ad473da9ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7bf9ba57c4c442a90180ed57eded8e9)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:14.439775 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221fb725-39ae-4295-a9bf-32ad473da9ba admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:14.557327 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-221fb725-39ae-4295-a9bf-32ad473da9ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7bf9ba57c4c442a90180ed57eded8e9', 'request_id': 'req-221fb725-39ae-4295-a9bf-32ad473da9ba'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:14.566645 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-221fb725-39ae-4295-a9bf-32ad473da9ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7bf9ba57c4c442a90180ed57eded8e9', 'request_id': 'req-221fb725-39ae-4295-a9bf-32ad473da9ba'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:14.654391 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-221fb725-39ae-4295-a9bf-32ad473da9ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7bf9ba57c4c442a90180ed57eded8e9', 'request_id': 'req-221fb725-39ae-4295-a9bf-32ad473da9ba'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:14.674283 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-221fb725-39ae-4295-a9bf-32ad473da9ba admin admin] The token cache is being invalidate because project d7bf9ba57c4c442a90180ed57eded8e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:34:14.674582 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-221fb725-39ae-4295-a9bf-32ad473da9ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:14.675279 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-221fb725-39ae-4295-a9bf-32ad473da9ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:34:14.676892 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1276/2589] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:34:14 2022] DELETE /identity/v3/projects/d7bf9ba57c4c442a90180ed57eded8e9 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:34:14.682172 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e295fd-9200-4721-85f7-83c8d1ae2ad8 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:14.682755 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e295fd-9200-4721-85f7-83c8d1ae2ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:14.682941 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e295fd-9200-4721-85f7-83c8d1ae2ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:14.709127 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-24e295fd-9200-4721-85f7-83c8d1ae2ad8 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:34:14.710936 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-24e295fd-9200-4721-85f7-83c8d1ae2ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:14.791467 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1314/2590] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:34:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:34:14.801149 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648eead1-4836-473b-955c-e8dbca4cc7a7 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:14.804382 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648eead1-4836-473b-955c-e8dbca4cc7a7 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:14.805093 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648eead1-4836-473b-955c-e8dbca4cc7a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:14.831057 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-648eead1-4836-473b-955c-e8dbca4cc7a7 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:34:14.832456 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-648eead1-4836-473b-955c-e8dbca4cc7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:14.841680 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1277/2591] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:34:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:34:14.846443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e295fd-9200-4721-85f7-83c8d1ae2ad8 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:14.859743 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e295fd-9200-4721-85f7-83c8d1ae2ad8 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:14.865980 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ede3bf-5a9a-4a5e-bb95-029564623b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:14.866937 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ede3bf-5a9a-4a5e-bb95-029564623b58 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:14.867726 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ede3bf-5a9a-4a5e-bb95-029564623b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:14.868071 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ede3bf-5a9a-4a5e-bb95-029564623b58 admin admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:14.868887 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ede3bf-5a9a-4a5e-bb95-029564623b58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:34:14.869261 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ede3bf-5a9a-4a5e-bb95-029564623b58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:14.871907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ede3bf-5a9a-4a5e-bb95-029564623b58 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:14.885263 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1315/2592] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:34:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:34:14.890750 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648eead1-4836-473b-955c-e8dbca4cc7a7 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:14.904000 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648eead1-4836-473b-955c-e8dbca4cc7a7 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:14.910077 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e092652-3dd9-4462-a39c-b69247faf397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:14.911205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e092652-3dd9-4462-a39c-b69247faf397 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:14.911542 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e092652-3dd9-4462-a39c-b69247faf397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:14.911871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e092652-3dd9-4462-a39c-b69247faf397 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:14.912830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e092652-3dd9-4462-a39c-b69247faf397 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:14.915402 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e092652-3dd9-4462-a39c-b69247faf397 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:14.996159 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1278/2593] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:34:14 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:34:15.000625 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ede3bf-5a9a-4a5e-bb95-029564623b58 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:15.014721 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ede3bf-5a9a-4a5e-bb95-029564623b58 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:15.084328 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f994516-fc2e-46a8-a1a7-917b3c83f77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:15.085102 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f994516-fc2e-46a8-a1a7-917b3c83f77c admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:15.085386 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f994516-fc2e-46a8-a1a7-917b3c83f77c admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:15.085672 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f994516-fc2e-46a8-a1a7-917b3c83f77c admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:15.086349 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f994516-fc2e-46a8-a1a7-917b3c83f77c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:15.088276 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f994516-fc2e-46a8-a1a7-917b3c83f77c admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:15.173700 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:34:15.179360 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1316/2594] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:34:14 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:34:15.185467 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e092652-3dd9-4462-a39c-b69247faf397 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:15.199780 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e092652-3dd9-4462-a39c-b69247faf397 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:15.204773 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ea4a2e-4693-43a4-bb4c-47220826d781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:15.205421 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea4a2e-4693-43a4-bb4c-47220826d781 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:15.205696 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea4a2e-4693-43a4-bb4c-47220826d781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:15.205969 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea4a2e-4693-43a4-bb4c-47220826d781 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:15.206683 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ea4a2e-4693-43a4-bb4c-47220826d781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:15.208341 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ea4a2e-4693-43a4-bb4c-47220826d781 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:15.223868 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1279/2595] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:34:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:34:15.228329 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f994516-fc2e-46a8-a1a7-917b3c83f77c admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:15.242264 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f994516-fc2e-46a8-a1a7-917b3c83f77c admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:15.248205 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87511d0b-861e-4fa4-bb28-5ae32afa849e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:15.248466 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87511d0b-861e-4fa4-bb28-5ae32afa849e admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:15.248763 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87511d0b-861e-4fa4-bb28-5ae32afa849e admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:15.249041 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87511d0b-861e-4fa4-bb28-5ae32afa849e admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:15.249759 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87511d0b-861e-4fa4-bb28-5ae32afa849e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:15.251409 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87511d0b-861e-4fa4-bb28-5ae32afa849e admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:15.264472 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1317/2596] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:34:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:34:15.269249 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ea4a2e-4693-43a4-bb4c-47220826d781 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:15.280003 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ea4a2e-4693-43a4-bb4c-47220826d781 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:15.285000 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d75e7-2c74-484d-ac9c-b71ee09c77b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:15.285688 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d75e7-2c74-484d-ac9c-b71ee09c77b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:15.285966 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d75e7-2c74-484d-ac9c-b71ee09c77b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:15.286233 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d75e7-2c74-484d-ac9c-b71ee09c77b3 admin admin] PATH_INFO: `/v3/projects/ca094dd6e4d64383b75640f040744140/users/7d37124881084b16a73f5eed20c70940/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:15.303353 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1d75e7-2c74-484d-ac9c-b71ee09c77b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca094dd6e4d64383b75640f040744140, user_id=7d37124881084b16a73f5eed20c70940, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:15.306023 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1d75e7-2c74-484d-ac9c-b71ee09c77b3 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:15.668860 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1280/2597] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:34:15 2022] PUT /identity/v3/projects/ca094dd6e4d64383b75640f040744140/users/7d37124881084b16a73f5eed20c70940/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:34:15.674188 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f6531-4bf1-4f97-9467-bbc53a37ba65 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:15.674455 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f6531-4bf1-4f97-9467-bbc53a37ba65 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:15.674791 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f6531-4bf1-4f97-9467-bbc53a37ba65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:15.694132 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-493f6531-4bf1-4f97-9467-bbc53a37ba65 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:34:15.695337 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-493f6531-4bf1-4f97-9467-bbc53a37ba65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:15.757181 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1318/2598] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:34:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:34:15.766825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d75e7-2c74-484d-ac9c-b71ee09c77b3 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:15.768514 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-1a1d75e7-2c74-484d-ac9c-b71ee09c77b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:15.810925 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:34:15.831436 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d75e7-2c74-484d-ac9c-b71ee09c77b3 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:15.835109 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f63ef5-530d-499f-82c3-fc4dc9a45c8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:15.835721 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f63ef5-530d-499f-82c3-fc4dc9a45c8d service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:15.836056 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f63ef5-530d-499f-82c3-fc4dc9a45c8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:15.836326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f63ef5-530d-499f-82c3-fc4dc9a45c8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:15.846838 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f63ef5-530d-499f-82c3-fc4dc9a45c8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:15.848268 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f63ef5-530d-499f-82c3-fc4dc9a45c8d service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:15.859211 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1281/2599] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:34:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:34:22.110657 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a0535-8cf4-4990-aa46-f4bd9b847552 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:22.111186 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a0535-8cf4-4990-aa46-f4bd9b847552 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:22.111186 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a0535-8cf4-4990-aa46-f4bd9b847552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:22.154826 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-cc0a0535-8cf4-4990-aa46-f4bd9b847552 None None] MFA Rules not processed for user `7d37124881084b16a73f5eed20c70940`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:34:22.156326 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-cc0a0535-8cf4-4990-aa46-f4bd9b847552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:22.267593 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1319/2600] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:34:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:34:22.272067 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f63ef5-530d-499f-82c3-fc4dc9a45c8d service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:22.283750 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f63ef5-530d-499f-82c3-fc4dc9a45c8d service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:22.288808 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8654841-a8d7-4f36-bd63-0cba99e5675b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:22.289922 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8654841-a8d7-4f36-bd63-0cba99e5675b admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:22.290528 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8654841-a8d7-4f36-bd63-0cba99e5675b admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:22.290961 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8654841-a8d7-4f36-bd63-0cba99e5675b admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:22.292652 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8654841-a8d7-4f36-bd63-0cba99e5675b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:22.294853 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8654841-a8d7-4f36-bd63-0cba99e5675b admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:22.380370 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1282/2601] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:34:22 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:34:22.384602 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a0535-8cf4-4990-aa46-f4bd9b847552 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:22.396285 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a0535-8cf4-4990-aa46-f4bd9b847552 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:22.459685 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94613ed5-600f-439e-b5b7-6cdeacbcc5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:22.460306 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94613ed5-600f-439e-b5b7-6cdeacbcc5b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:22.460620 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94613ed5-600f-439e-b5b7-6cdeacbcc5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:22.460894 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94613ed5-600f-439e-b5b7-6cdeacbcc5b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:22.461569 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94613ed5-600f-439e-b5b7-6cdeacbcc5b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:22.463483 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94613ed5-600f-439e-b5b7-6cdeacbcc5b5 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:22.527808 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1320/2602] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:34:22 2022] POST /identity/v3/users => generated 299 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:34:22.531560 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8654841-a8d7-4f36-bd63-0cba99e5675b admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:22.543021 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8654841-a8d7-4f36-bd63-0cba99e5675b admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:22.548236 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289809bd-4fbd-460f-b642-4bb228cf7626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:22.548951 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289809bd-4fbd-460f-b642-4bb228cf7626 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:22.549416 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289809bd-4fbd-460f-b642-4bb228cf7626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:22.549836 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289809bd-4fbd-460f-b642-4bb228cf7626 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:22.550808 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289809bd-4fbd-460f-b642-4bb228cf7626 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:22.552762 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289809bd-4fbd-460f-b642-4bb228cf7626 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:22.565905 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1283/2603] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:34:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:34:22.569831 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94613ed5-600f-439e-b5b7-6cdeacbcc5b5 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:22.596749 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94613ed5-600f-439e-b5b7-6cdeacbcc5b5 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:22.601608 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75671893-aa65-414c-b0e6-4a57fa9f63d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:22.602335 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75671893-aa65-414c-b0e6-4a57fa9f63d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:22.602616 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75671893-aa65-414c-b0e6-4a57fa9f63d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:22.602891 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75671893-aa65-414c-b0e6-4a57fa9f63d5 admin admin] PATH_INFO: `/v3/projects/051c21501570451e966a6cd64dc5b7be/users/9457d9aede4247cc93c5a7aabc7984ec/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:22.607632 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:34:22.619799 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75671893-aa65-414c-b0e6-4a57fa9f63d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=051c21501570451e966a6cd64dc5b7be, user_id=9457d9aede4247cc93c5a7aabc7984ec, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:22.622080 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75671893-aa65-414c-b0e6-4a57fa9f63d5 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:22.690902 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1321/2604] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:34:22 2022] PUT /identity/v3/projects/051c21501570451e966a6cd64dc5b7be/users/9457d9aede4247cc93c5a7aabc7984ec/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:34:22.695030 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289809bd-4fbd-460f-b642-4bb228cf7626 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:22.705418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289809bd-4fbd-460f-b642-4bb228cf7626 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:22.766537 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1fc3b-0dd2-4055-81f3-93e56671009e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:22.767191 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1fc3b-0dd2-4055-81f3-93e56671009e admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:22.767476 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1fc3b-0dd2-4055-81f3-93e56671009e admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:22.767747 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1fc3b-0dd2-4055-81f3-93e56671009e admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:22.768476 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c1fc3b-0dd2-4055-81f3-93e56671009e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:22.770125 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c1fc3b-0dd2-4055-81f3-93e56671009e admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:22.785358 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1284/2605] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:34:22 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:34:22.786260 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75671893-aa65-414c-b0e6-4a57fa9f63d5 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:22.798395 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75671893-aa65-414c-b0e6-4a57fa9f63d5 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:22.803589 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d9647-e53c-494a-ac7f-2f762c59af14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:22.804335 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d9647-e53c-494a-ac7f-2f762c59af14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:22.804664 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d9647-e53c-494a-ac7f-2f762c59af14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:22.804968 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d9647-e53c-494a-ac7f-2f762c59af14 admin admin] PATH_INFO: `/v3/projects/051c21501570451e966a6cd64dc5b7be/users/9457d9aede4247cc93c5a7aabc7984ec/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:22.806940 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596d9647-e53c-494a-ac7f-2f762c59af14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=051c21501570451e966a6cd64dc5b7be, user_id=9457d9aede4247cc93c5a7aabc7984ec, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:22.809694 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596d9647-e53c-494a-ac7f-2f762c59af14 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:22.820503 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1322/2606] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:34:22 2022] PUT /identity/v3/projects/051c21501570451e966a6cd64dc5b7be/users/9457d9aede4247cc93c5a7aabc7984ec/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:34:24.263224 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1fc3b-0dd2-4055-81f3-93e56671009e admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:24.265067 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e5c1fc3b-0dd2-4055-81f3-93e56671009e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:24.413022 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1fc3b-0dd2-4055-81f3-93e56671009e admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:24.436799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:34:24.482211 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3634fcd-a332-4231-8102-833cea712d5b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff761f6f81a46098f94bd2ec2be9c1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:24.482945 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3634fcd-a332-4231-8102-833cea712d5b service placement] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:24.483285 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3634fcd-a332-4231-8102-833cea712d5b service placement] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:24.483614 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3634fcd-a332-4231-8102-833cea712d5b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:24.485799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f3634fcd-a332-4231-8102-833cea712d5b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:24.498961 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3634fcd-a332-4231-8102-833cea712d5b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:24.500800 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3634fcd-a332-4231-8102-833cea712d5b service placement] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:24.512208 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1285/2607] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:34:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:34:28.462210 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c082692-510e-4473-804f-39ba8a549c70 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:28.463026 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c082692-510e-4473-804f-39ba8a549c70 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:28.463302 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c082692-510e-4473-804f-39ba8a549c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:28.495972 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-6c082692-510e-4473-804f-39ba8a549c70 None None] MFA Rules not processed for user `9457d9aede4247cc93c5a7aabc7984ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:34:28.497598 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-6c082692-510e-4473-804f-39ba8a549c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:28.622091 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1323/2608] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:34:28 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:34:28.634728 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3634fcd-a332-4231-8102-833cea712d5b service placement] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:28.638042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f3634fcd-a332-4231-8102-833cea712d5b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:28.709938 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3634fcd-a332-4231-8102-833cea712d5b service placement] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:28.714532 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e9c61a-12b1-4fd3-94c5-5308cf43096f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1ce2a2b4eb405b999d3fb756795176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:28.715184 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e9c61a-12b1-4fd3-94c5-5308cf43096f service glance] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:28.715729 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e9c61a-12b1-4fd3-94c5-5308cf43096f service glance] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:28.716007 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e9c61a-12b1-4fd3-94c5-5308cf43096f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:28.716806 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3698f57-352c-4f33-bb8f-32fc43cecfb6 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:28.717341 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3698f57-352c-4f33-bb8f-32fc43cecfb6 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:28.717811 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3698f57-352c-4f33-bb8f-32fc43cecfb6 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:28.719775 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1324/2609] 10.222.0.26 () {54 vars in 921 bytes} [Fri Aug 5 14:34:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 14:34:28.729773 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe5cad-740d-4ed5-8814-beef953dde56 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:28.730053 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe5cad-740d-4ed5-8814-beef953dde56 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:28.730513 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe5cad-740d-4ed5-8814-beef953dde56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:28.768845 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-fdbe5cad-740d-4ed5-8814-beef953dde56 None None] MFA Rules not processed for user `5ec4940c99fc4fa8a5bc7cdabc4558fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:34:28.770387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-fdbe5cad-740d-4ed5-8814-beef953dde56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:28.822603 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e9c61a-12b1-4fd3-94c5-5308cf43096f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:28.826210 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e9c61a-12b1-4fd3-94c5-5308cf43096f service glance] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:28.843730 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1286/2610] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:34:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 14:34:28.954824 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1325/2611] 10.222.0.26 () {58 vars in 1018 bytes} [Fri Aug 5 14:34:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:34:28.974283 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e9c61a-12b1-4fd3-94c5-5308cf43096f service glance] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:28.976449 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e1e9c61a-12b1-4fd3-94c5-5308cf43096f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:28.992873 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e9c61a-12b1-4fd3-94c5-5308cf43096f service glance] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:28.999367 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a485310f-6bbf-4598-a269-17c828e0b357 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1ce2a2b4eb405b999d3fb756795176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:29.001757 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485310f-6bbf-4598-a269-17c828e0b357 service glance] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:29.002428 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485310f-6bbf-4598-a269-17c828e0b357 service glance] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:29.002921 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485310f-6bbf-4598-a269-17c828e0b357 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:29.050252 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a485310f-6bbf-4598-a269-17c828e0b357 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:29.054016 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a485310f-6bbf-4598-a269-17c828e0b357 service glance] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:29.082173 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1287/2612] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:34:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:34:29.190417 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444456a-f722-49d6-b8d7-29bb704ae0ab None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:29.190702 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444456a-f722-49d6-b8d7-29bb704ae0ab None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:29.190975 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444456a-f722-49d6-b8d7-29bb704ae0ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:29.227752 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-d444456a-f722-49d6-b8d7-29bb704ae0ab None None] MFA Rules not processed for user `17e3b509a2de4159bf234594573f5e18`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:34:29.229135 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-d444456a-f722-49d6-b8d7-29bb704ae0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:29.260145 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:34:29.293849 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1326/2613] 10.222.0.26 () {58 vars in 1009 bytes} [Fri Aug 5 14:34:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:34:29.305964 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a485310f-6bbf-4598-a269-17c828e0b357 service glance] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:29.307654 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a485310f-6bbf-4598-a269-17c828e0b357 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:29.356434 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee73c4a-5390-41fe-aaae-b7af1c156517 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:29.357554 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee73c4a-5390-41fe-aaae-b7af1c156517 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:29.358039 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee73c4a-5390-41fe-aaae-b7af1c156517 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:29.360129 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1327/2614] 10.222.0.26 () {54 vars in 921 bytes} [Fri Aug 5 14:34:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:34:29.365510 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e788420c-31d9-4c6c-a52c-b7a822b4ca5d None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:29.367825 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e788420c-31d9-4c6c-a52c-b7a822b4ca5d None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:29.367825 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e788420c-31d9-4c6c-a52c-b7a822b4ca5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:29.374639 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:34:29.389233 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-e788420c-31d9-4c6c-a52c-b7a822b4ca5d None None] MFA Rules not processed for user `695e37560f8f495e987d1bd17acc2680`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:34:29.390451 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-e788420c-31d9-4c6c-a52c-b7a822b4ca5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:29.392758 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a485310f-6bbf-4598-a269-17c828e0b357 service glance] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:29.398947 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eab90b-2bd4-4971-aa2a-7df42120023a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dddd7eea67f34f0cbb5f6a1cd85f1fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:29.399657 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eab90b-2bd4-4971-aa2a-7df42120023a service swift] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:29.400009 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eab90b-2bd4-4971-aa2a-7df42120023a service swift] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:29.400349 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eab90b-2bd4-4971-aa2a-7df42120023a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:29.410658 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eab90b-2bd4-4971-aa2a-7df42120023a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:29.412262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eab90b-2bd4-4971-aa2a-7df42120023a service swift] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:29.425930 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1288/2615] 10.222.0.26 () {60 vars in 1384 bytes} [Fri Aug 5 14:34:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 05 14:34:29.450179 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1328/2616] 10.222.0.26 () {58 vars in 1018 bytes} [Fri Aug 5 14:34:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:34:29.465855 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eab90b-2bd4-4971-aa2a-7df42120023a service swift] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:29.467340 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-07eab90b-2bd4-4971-aa2a-7df42120023a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:29.482886 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eab90b-2bd4-4971-aa2a-7df42120023a service swift] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:29.487148 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c8b25-55f4-4ac3-b847-5997f978153a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:29.487768 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c8b25-55f4-4ac3-b847-5997f978153a service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:29.488041 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c8b25-55f4-4ac3-b847-5997f978153a service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:29.488311 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c8b25-55f4-4ac3-b847-5997f978153a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:29.497290 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c8b25-55f4-4ac3-b847-5997f978153a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:29.498759 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c8b25-55f4-4ac3-b847-5997f978153a service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:29.507460 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1289/2617] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:34:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:34:29.640449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1a56f-1a71-4555-9900-d9a1269d2871 None None] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:29.640932 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1a56f-1a71-4555-9900-d9a1269d2871 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:29.641197 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1a56f-1a71-4555-9900-d9a1269d2871 None None] PATH_INFO: `/` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:29.643229 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1329/2618] 10.222.0.26 () {54 vars in 921 bytes} [Fri Aug 5 14:34:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 14:34:29.649272 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af6e990-f60a-4a9b-a0f9-866e57848404 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:29.649810 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af6e990-f60a-4a9b-a0f9-866e57848404 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:29.650033 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af6e990-f60a-4a9b-a0f9-866e57848404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:29.678589 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-5af6e990-f60a-4a9b-a0f9-866e57848404 None None] MFA Rules not processed for user `5ec4940c99fc4fa8a5bc7cdabc4558fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:34:29.680312 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5af6e990-f60a-4a9b-a0f9-866e57848404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:29.707034 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1290/2619] 10.222.0.26 () {58 vars in 1018 bytes} [Fri Aug 5 14:34:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:34:29.720095 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1a56f-1a71-4555-9900-d9a1269d2871 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:29.739043 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1a56f-1a71-4555-9900-d9a1269d2871 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:29.750432 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7673006-bf83-4ed5-8c4e-0b3a41010c87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:29.753190 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7673006-bf83-4ed5-8c4e-0b3a41010c87 service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:29.753657 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7673006-bf83-4ed5-8c4e-0b3a41010c87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:29.754164 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7673006-bf83-4ed5-8c4e-0b3a41010c87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:29.781890 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7673006-bf83-4ed5-8c4e-0b3a41010c87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:29.784927 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7673006-bf83-4ed5-8c4e-0b3a41010c87 service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:29.801658 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1330/2620] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:34:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:34:32.622383 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa1ee6-a2ca-40bb-bae8-013568dd9f80 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:32.622933 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa1ee6-a2ca-40bb-bae8-013568dd9f80 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:32.623164 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa1ee6-a2ca-40bb-bae8-013568dd9f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:32.648384 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-55aa1ee6-a2ca-40bb-bae8-013568dd9f80 None None] MFA Rules not processed for user `17e3b509a2de4159bf234594573f5e18`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:34:32.649592 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-55aa1ee6-a2ca-40bb-bae8-013568dd9f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:32.658255 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1291/2621] 10.222.0.26 () {58 vars in 1009 bytes} [Fri Aug 5 14:34:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:34:32.686062 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7673006-bf83-4ed5-8c4e-0b3a41010c87 service neutron] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:32.706066 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7673006-bf83-4ed5-8c4e-0b3a41010c87 service neutron] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:32.718131 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ed09ae-ce9a-465f-8df5-f5569e8ff41d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dddd7eea67f34f0cbb5f6a1cd85f1fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:32.719251 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ed09ae-ce9a-465f-8df5-f5569e8ff41d service swift] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:32.719652 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ed09ae-ce9a-465f-8df5-f5569e8ff41d service swift] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:32.720035 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ed09ae-ce9a-465f-8df5-f5569e8ff41d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:32.741634 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ed09ae-ce9a-465f-8df5-f5569e8ff41d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:32.743747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ed09ae-ce9a-465f-8df5-f5569e8ff41d service swift] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:32.776758 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1331/2622] 10.222.0.26 () {60 vars in 1384 bytes} [Fri Aug 5 14:34:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 14:34:42.152475 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78b20c-f540-4b4b-9d53-b649098af961 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:42.153012 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78b20c-f540-4b4b-9d53-b649098af961 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:42.153012 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78b20c-f540-4b4b-9d53-b649098af961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:42.173323 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-9b78b20c-f540-4b4b-9d53-b649098af961 None None] MFA Rules not processed for user `7d37124881084b16a73f5eed20c70940`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:34:42.174550 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-9b78b20c-f540-4b4b-9d53-b649098af961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:42.181345 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1292/2623] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:34:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:34:42.753046 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=47 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:34:49.007649 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a6d638-46d8-492d-bf68-6049dc5060fe None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:49.008271 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a6d638-46d8-492d-bf68-6049dc5060fe None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:49.008271 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a6d638-46d8-492d-bf68-6049dc5060fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:49.049826 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-04a6d638-46d8-492d-bf68-6049dc5060fe None None] MFA Rules not processed for user `17e3b509a2de4159bf234594573f5e18`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:34:49.054369 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-04a6d638-46d8-492d-bf68-6049dc5060fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:34:49.092591 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1332/2624] 10.222.0.26 () {58 vars in 1009 bytes} [Fri Aug 5 14:34:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:34:49.112946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78b20c-f540-4b4b-9d53-b649098af961 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:34:49.153295 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78b20c-f540-4b4b-9d53-b649098af961 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:34:49.153295 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4cc4ae-e836-4960-8568-82c47fe9f856 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dddd7eea67f34f0cbb5f6a1cd85f1fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:34:49.153295 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4cc4ae-e836-4960-8568-82c47fe9f856 service swift] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:34:49.153295 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4cc4ae-e836-4960-8568-82c47fe9f856 service swift] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:34:49.153944 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4cc4ae-e836-4960-8568-82c47fe9f856 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:34:49.170578 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4cc4ae-e836-4960-8568-82c47fe9f856 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:34:49.177435 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4cc4ae-e836-4960-8568-82c47fe9f856 service swift] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:34:49.214924 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1293/2625] 10.222.0.26 () {60 vars in 1384 bytes} [Fri Aug 5 14:34:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 14:34:59.037763 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=3 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:34:59.194590 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=28 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:35:04.302870 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f92ea-835a-4322-8b64-74fcc389cf1f None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:35:04.303327 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f92ea-835a-4322-8b64-74fcc389cf1f None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:35:04.303327 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f92ea-835a-4322-8b64-74fcc389cf1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:35:04.323417 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-3e9f92ea-835a-4322-8b64-74fcc389cf1f None None] MFA Rules not processed for user `17e3b509a2de4159bf234594573f5e18`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:35:04.325262 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-3e9f92ea-835a-4322-8b64-74fcc389cf1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:35:04.333079 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1333/2626] 10.222.0.26 () {58 vars in 967 bytes} [Fri Aug 5 14:35:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:35:04.346119 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4cc4ae-e836-4960-8568-82c47fe9f856 service swift] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:35:04.358939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4cc4ae-e836-4960-8568-82c47fe9f856 service swift] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:35:04.364561 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7398c9c-9652-4093-8ab3-cb739a33cec2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dddd7eea67f34f0cbb5f6a1cd85f1fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:35:04.365170 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7398c9c-9652-4093-8ab3-cb739a33cec2 service swift] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:35:04.365631 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7398c9c-9652-4093-8ab3-cb739a33cec2 service swift] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:35:04.366308 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7398c9c-9652-4093-8ab3-cb739a33cec2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:35:04.377366 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7398c9c-9652-4093-8ab3-cb739a33cec2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:35:04.379084 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7398c9c-9652-4093-8ab3-cb739a33cec2 service swift] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:35:04.388732 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1294/2627] 10.222.0.26 () {60 vars in 1384 bytes} [Fri Aug 5 14:35:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 14:35:10.799224 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f92ea-835a-4322-8b64-74fcc389cf1f None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:35:10.808514 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f92ea-835a-4322-8b64-74fcc389cf1f None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:35:10.812666 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e0581-2769-4892-9d7e-eb85a21f0530 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:35:10.813356 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e0581-2769-4892-9d7e-eb85a21f0530 service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:35:10.813634 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e0581-2769-4892-9d7e-eb85a21f0530 service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:35:10.813907 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e0581-2769-4892-9d7e-eb85a21f0530 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:35:10.815757 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-b12e0581-2769-4892-9d7e-eb85a21f0530 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:35:10.879004 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12e0581-2769-4892-9d7e-eb85a21f0530 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:35:10.880567 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12e0581-2769-4892-9d7e-eb85a21f0530 service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:35:10.891310 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1334/2628] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:35:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:35:11.191799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7398c9c-9652-4093-8ab3-cb739a33cec2 service swift] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:35:11.201595 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7398c9c-9652-4093-8ab3-cb739a33cec2 service swift] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:35:11.206191 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4a71c-7816-4ce9-aa26-dc9d87f92eb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:35:11.206877 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4a71c-7816-4ce9-aa26-dc9d87f92eb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:35:11.207222 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4a71c-7816-4ce9-aa26-dc9d87f92eb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:35:11.207667 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4a71c-7816-4ce9-aa26-dc9d87f92eb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:35:11.209486 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-b0f4a71c-7816-4ce9-aa26-dc9d87f92eb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:35:11.223588 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f4a71c-7816-4ce9-aa26-dc9d87f92eb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:35:11.225096 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f4a71c-7816-4ce9-aa26-dc9d87f92eb2 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:35:11.236498 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1295/2629] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:35:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:35:14.334634 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e0581-2769-4892-9d7e-eb85a21f0530 service neutron] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:35:14.336068 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-b12e0581-2769-4892-9d7e-eb85a21f0530 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:35:14.352534 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e0581-2769-4892-9d7e-eb85a21f0530 service neutron] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:35:14.357539 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2712cde-f46b-4a43-b92c-e98ca8703304 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff761f6f81a46098f94bd2ec2be9c1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:35:14.358202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2712cde-f46b-4a43-b92c-e98ca8703304 service placement] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:35:14.358544 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2712cde-f46b-4a43-b92c-e98ca8703304 service placement] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:35:14.358989 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2712cde-f46b-4a43-b92c-e98ca8703304 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:35:14.360779 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-a2712cde-f46b-4a43-b92c-e98ca8703304 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:35:14.373189 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2712cde-f46b-4a43-b92c-e98ca8703304 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:35:14.375522 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2712cde-f46b-4a43-b92c-e98ca8703304 service placement] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:35:14.386476 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1335/2630] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:35:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:35:18.104606 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4a71c-7816-4ce9-aa26-dc9d87f92eb2 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:35:18.106021 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-b0f4a71c-7816-4ce9-aa26-dc9d87f92eb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:35:18.176653 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4a71c-7816-4ce9-aa26-dc9d87f92eb2 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:35:18.183413 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ec708-80ab-45f3-92bb-4d74f779d534 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:35:18.183413 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ec708-80ab-45f3-92bb-4d74f779d534 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:35:18.183413 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ec708-80ab-45f3-92bb-4d74f779d534 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:35:18.183413 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ec708-80ab-45f3-92bb-4d74f779d534 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:35:18.184619 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-414ec708-80ab-45f3-92bb-4d74f779d534 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:35:18.201444 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ec708-80ab-45f3-92bb-4d74f779d534 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:35:18.203044 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ec708-80ab-45f3-92bb-4d74f779d534 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:35:18.215023 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1296/2631] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:35:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:35:18.329729 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2712cde-f46b-4a43-b92c-e98ca8703304 service placement] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:35:18.331202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-a2712cde-f46b-4a43-b92c-e98ca8703304 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:35:18.345522 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2712cde-f46b-4a43-b92c-e98ca8703304 service placement] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:35:18.350078 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c2611-f4c9-42a5-8eb8-39868e9e6d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:35:18.350714 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c2611-f4c9-42a5-8eb8-39868e9e6d8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:35:18.356250 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c2611-f4c9-42a5-8eb8-39868e9e6d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:35:18.356250 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c2611-f4c9-42a5-8eb8-39868e9e6d8e admin admin] PATH_INFO: `/v3/users/33b2cb4c04f6495bbb5ff171df953ef0` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:35:18.356250 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971c2611-f4c9-42a5-8eb8-39868e9e6d8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=33b2cb4c04f6495bbb5ff171df953ef0)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:35:18.356250 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971c2611-f4c9-42a5-8eb8-39868e9e6d8e admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:35:18.573801 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-971c2611-f4c9-42a5-8eb8-39868e9e6d8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33b2cb4c04f6495bbb5ff171df953ef0'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:35:18.583029 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-971c2611-f4c9-42a5-8eb8-39868e9e6d8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33b2cb4c04f6495bbb5ff171df953ef0'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:35:18.583834 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-971c2611-f4c9-42a5-8eb8-39868e9e6d8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33b2cb4c04f6495bbb5ff171df953ef0'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:35:18.584464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-971c2611-f4c9-42a5-8eb8-39868e9e6d8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33b2cb4c04f6495bbb5ff171df953ef0'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:35:18.604836 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-971c2611-f4c9-42a5-8eb8-39868e9e6d8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33b2cb4c04f6495bbb5ff171df953ef0'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:35:18.708052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1336/2632] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:35:18 2022] DELETE /identity/v3/users/33b2cb4c04f6495bbb5ff171df953ef0 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:35:19.162753 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ec708-80ab-45f3-92bb-4d74f779d534 service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:35:19.172305 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ec708-80ab-45f3-92bb-4d74f779d534 service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:35:19.243197 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059e165-5624-4d0d-b9cf-44c29ab051d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:35:19.243850 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059e165-5624-4d0d-b9cf-44c29ab051d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:35:19.244127 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059e165-5624-4d0d-b9cf-44c29ab051d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:35:19.244394 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059e165-5624-4d0d-b9cf-44c29ab051d3 admin admin] PATH_INFO: `/v3/projects/4717d08ca01041088bd4b95f00d7b5ef` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:35:19.245579 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0059e165-5624-4d0d-b9cf-44c29ab051d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4717d08ca01041088bd4b95f00d7b5ef)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:35:19.247634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0059e165-5624-4d0d-b9cf-44c29ab051d3 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:35:19.355959 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-0059e165-5624-4d0d-b9cf-44c29ab051d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4717d08ca01041088bd4b95f00d7b5ef', 'request_id': 'req-0059e165-5624-4d0d-b9cf-44c29ab051d3'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:35:19.362595 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-0059e165-5624-4d0d-b9cf-44c29ab051d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4717d08ca01041088bd4b95f00d7b5ef', 'request_id': 'req-0059e165-5624-4d0d-b9cf-44c29ab051d3'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:35:19.420482 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-0059e165-5624-4d0d-b9cf-44c29ab051d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4717d08ca01041088bd4b95f00d7b5ef', 'request_id': 'req-0059e165-5624-4d0d-b9cf-44c29ab051d3'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:35:19.433996 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-0059e165-5624-4d0d-b9cf-44c29ab051d3 admin admin] The token cache is being invalidate because project 4717d08ca01041088bd4b95f00d7b5ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:35:19.434408 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-0059e165-5624-4d0d-b9cf-44c29ab051d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:35:19.435214 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-0059e165-5624-4d0d-b9cf-44c29ab051d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:35:19.438155 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1297/2633] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:35:19 2022] DELETE /identity/v3/projects/4717d08ca01041088bd4b95f00d7b5ef => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:35:19.442163 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c2611-f4c9-42a5-8eb8-39868e9e6d8e admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:35:19.443749 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-971c2611-f4c9-42a5-8eb8-39868e9e6d8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:35:19.458357 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c2611-f4c9-42a5-8eb8-39868e9e6d8e admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:35:19.519254 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:35:19.540582 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaba270-f16e-4280-9c34-26684657e955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:35:19.541602 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaba270-f16e-4280-9c34-26684657e955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:35:19.541903 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaba270-f16e-4280-9c34-26684657e955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:35:19.542670 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaba270-f16e-4280-9c34-26684657e955 admin admin] PATH_INFO: `/v3/users/2806cf9ecdcb410aad9d0c4710ae43dc` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:35:19.543963 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaba270-f16e-4280-9c34-26684657e955 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2806cf9ecdcb410aad9d0c4710ae43dc)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:35:19.551872 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaba270-f16e-4280-9c34-26684657e955 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:35:19.776512 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-baaba270-f16e-4280-9c34-26684657e955 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2806cf9ecdcb410aad9d0c4710ae43dc'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:35:19.788760 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-baaba270-f16e-4280-9c34-26684657e955 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2806cf9ecdcb410aad9d0c4710ae43dc'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:35:19.789496 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-baaba270-f16e-4280-9c34-26684657e955 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2806cf9ecdcb410aad9d0c4710ae43dc'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:35:19.790622 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-baaba270-f16e-4280-9c34-26684657e955 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2806cf9ecdcb410aad9d0c4710ae43dc'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:35:19.807342 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-baaba270-f16e-4280-9c34-26684657e955 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2806cf9ecdcb410aad9d0c4710ae43dc'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:35:19.879274 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1337/2634] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:35:19 2022] DELETE /identity/v3/users/2806cf9ecdcb410aad9d0c4710ae43dc => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:35:20.159910 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059e165-5624-4d0d-b9cf-44c29ab051d3 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:35:20.161113 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-0059e165-5624-4d0d-b9cf-44c29ab051d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:35:20.176760 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059e165-5624-4d0d-b9cf-44c29ab051d3 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:35:20.181360 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:35:20.242239 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c29cd-1cbe-4098-bf05-302f0e368158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:35:20.247319 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c29cd-1cbe-4098-bf05-302f0e368158 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:35:20.247319 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c29cd-1cbe-4098-bf05-302f0e368158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:35:20.247319 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c29cd-1cbe-4098-bf05-302f0e368158 admin admin] PATH_INFO: `/v3/projects/ccaa8dabff724f9ca0b296a92b4bfc70` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:35:20.247319 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c29cd-1cbe-4098-bf05-302f0e368158 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccaa8dabff724f9ca0b296a92b4bfc70)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:35:20.247319 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c29cd-1cbe-4098-bf05-302f0e368158 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:35:20.365263 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a39c29cd-1cbe-4098-bf05-302f0e368158 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccaa8dabff724f9ca0b296a92b4bfc70', 'request_id': 'req-a39c29cd-1cbe-4098-bf05-302f0e368158'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:35:20.372110 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a39c29cd-1cbe-4098-bf05-302f0e368158 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccaa8dabff724f9ca0b296a92b4bfc70', 'request_id': 'req-a39c29cd-1cbe-4098-bf05-302f0e368158'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:35:20.427354 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a39c29cd-1cbe-4098-bf05-302f0e368158 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccaa8dabff724f9ca0b296a92b4bfc70', 'request_id': 'req-a39c29cd-1cbe-4098-bf05-302f0e368158'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:35:20.439219 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a39c29cd-1cbe-4098-bf05-302f0e368158 admin admin] The token cache is being invalidate because project ccaa8dabff724f9ca0b296a92b4bfc70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:35:20.439525 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a39c29cd-1cbe-4098-bf05-302f0e368158 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:35:20.440310 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a39c29cd-1cbe-4098-bf05-302f0e368158 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:35:20.441880 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1298/2635] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:35:20 2022] DELETE /identity/v3/projects/ccaa8dabff724f9ca0b296a92b4bfc70 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:35:26.492017 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaba270-f16e-4280-9c34-26684657e955 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:35:26.493476 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-baaba270-f16e-4280-9c34-26684657e955 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:35:26.555174 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaba270-f16e-4280-9c34-26684657e955 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:35:26.603389 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ade54-d2ee-4262-8e69-6d4cbaeca193 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:35:26.604026 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ade54-d2ee-4262-8e69-6d4cbaeca193 service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:35:26.604305 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ade54-d2ee-4262-8e69-6d4cbaeca193 service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:35:26.604606 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ade54-d2ee-4262-8e69-6d4cbaeca193 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:35:26.606378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-387ade54-d2ee-4262-8e69-6d4cbaeca193 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:35:26.643518 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:35:26.671841 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ade54-d2ee-4262-8e69-6d4cbaeca193 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:35:26.678576 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ade54-d2ee-4262-8e69-6d4cbaeca193 service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:35:26.683319 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1338/2636] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:35:26 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:35:27.049597 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c29cd-1cbe-4098-bf05-302f0e368158 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:35:27.050672 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a39c29cd-1cbe-4098-bf05-302f0e368158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:35:27.063861 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c29cd-1cbe-4098-bf05-302f0e368158 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:35:27.069174 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31bc5ac-37cc-4b85-9d34-e9a492875d12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:35:27.069902 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31bc5ac-37cc-4b85-9d34-e9a492875d12 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:35:27.070237 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31bc5ac-37cc-4b85-9d34-e9a492875d12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:35:27.070512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31bc5ac-37cc-4b85-9d34-e9a492875d12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:35:27.072327 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-b31bc5ac-37cc-4b85-9d34-e9a492875d12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:35:27.146038 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31bc5ac-37cc-4b85-9d34-e9a492875d12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:35:27.147503 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31bc5ac-37cc-4b85-9d34-e9a492875d12 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:35:27.159292 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1299/2637] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:35:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:35:31.449681 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ade54-d2ee-4262-8e69-6d4cbaeca193 service neutron] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:35:31.451126 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-387ade54-d2ee-4262-8e69-6d4cbaeca193 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:35:31.464784 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ade54-d2ee-4262-8e69-6d4cbaeca193 service neutron] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:35:31.470215 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446861d3-4c28-48f9-a72b-ac6157e0c43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:35:31.470866 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446861d3-4c28-48f9-a72b-ac6157e0c43a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:35:31.478286 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446861d3-4c28-48f9-a72b-ac6157e0c43a admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:35:31.478286 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446861d3-4c28-48f9-a72b-ac6157e0c43a admin admin] PATH_INFO: `/v3/users/99dc94fff06f42c39f3750484f6d8e9d` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:35:31.486010 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446861d3-4c28-48f9-a72b-ac6157e0c43a admin admin] RBAC: Authorizing `identity:delete_user(user_id=99dc94fff06f42c39f3750484f6d8e9d)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:35:31.487941 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446861d3-4c28-48f9-a72b-ac6157e0c43a admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:35:31.697449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-446861d3-4c28-48f9-a72b-ac6157e0c43a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99dc94fff06f42c39f3750484f6d8e9d'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:35:31.705221 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-446861d3-4c28-48f9-a72b-ac6157e0c43a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99dc94fff06f42c39f3750484f6d8e9d'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:35:31.705956 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-446861d3-4c28-48f9-a72b-ac6157e0c43a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99dc94fff06f42c39f3750484f6d8e9d'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:35:31.706567 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-446861d3-4c28-48f9-a72b-ac6157e0c43a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99dc94fff06f42c39f3750484f6d8e9d'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:35:31.724402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-446861d3-4c28-48f9-a72b-ac6157e0c43a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99dc94fff06f42c39f3750484f6d8e9d'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:35:31.834769 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1339/2638] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:35:31 2022] DELETE /identity/v3/users/99dc94fff06f42c39f3750484f6d8e9d => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:35:32.109283 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31bc5ac-37cc-4b85-9d34-e9a492875d12 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:35:32.121466 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31bc5ac-37cc-4b85-9d34-e9a492875d12 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:35:32.155978 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:35:32.183865 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c287b-ad54-4f18-b319-7ba725e6efba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:35:32.184544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c287b-ad54-4f18-b319-7ba725e6efba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:35:32.189314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c287b-ad54-4f18-b319-7ba725e6efba admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:35:32.189314 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c287b-ad54-4f18-b319-7ba725e6efba admin admin] PATH_INFO: `/v3/projects/0b15967673f14cf9a208a6143a4e87cc` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:35:32.197644 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425c287b-ad54-4f18-b319-7ba725e6efba admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b15967673f14cf9a208a6143a4e87cc)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:35:32.199563 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425c287b-ad54-4f18-b319-7ba725e6efba admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:35:32.323786 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-425c287b-ad54-4f18-b319-7ba725e6efba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b15967673f14cf9a208a6143a4e87cc', 'request_id': 'req-425c287b-ad54-4f18-b319-7ba725e6efba'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:35:32.331646 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-425c287b-ad54-4f18-b319-7ba725e6efba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b15967673f14cf9a208a6143a4e87cc', 'request_id': 'req-425c287b-ad54-4f18-b319-7ba725e6efba'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:35:32.447048 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-425c287b-ad54-4f18-b319-7ba725e6efba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b15967673f14cf9a208a6143a4e87cc', 'request_id': 'req-425c287b-ad54-4f18-b319-7ba725e6efba'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:35:32.475091 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-425c287b-ad54-4f18-b319-7ba725e6efba admin admin] The token cache is being invalidate because project 0b15967673f14cf9a208a6143a4e87cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:35:32.477344 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-425c287b-ad54-4f18-b319-7ba725e6efba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:35:32.478485 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-425c287b-ad54-4f18-b319-7ba725e6efba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:35:32.488929 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1300/2639] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:35:32 2022] DELETE /identity/v3/projects/0b15967673f14cf9a208a6143a4e87cc => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:35:32.489595 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba989b-cdc0-41b5-84fe-f6adc25eb763 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:35:32.490055 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba989b-cdc0-41b5-84fe-f6adc25eb763 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:35:32.490169 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba989b-cdc0-41b5-84fe-f6adc25eb763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:35:32.512920 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-03ba989b-cdc0-41b5-84fe-f6adc25eb763 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:35:32.515402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-03ba989b-cdc0-41b5-84fe-f6adc25eb763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:35:32.631153 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1340/2640] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:35:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:35:32.638996 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a6b1a-4c29-490a-a2b1-445a3b041ac0 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:35:32.639366 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a6b1a-4c29-490a-a2b1-445a3b041ac0 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:35:32.639742 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a6b1a-4c29-490a-a2b1-445a3b041ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:35:32.669768 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-bf2a6b1a-4c29-490a-a2b1-445a3b041ac0 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:35:32.671680 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-bf2a6b1a-4c29-490a-a2b1-445a3b041ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:35:32.681320 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1301/2641] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:35:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:35:32.685866 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba989b-cdc0-41b5-84fe-f6adc25eb763 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:35:32.696977 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba989b-cdc0-41b5-84fe-f6adc25eb763 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:35:32.704961 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f34449-3d70-4da5-a795-4ff3a7e311f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:35:32.711850 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f34449-3d70-4da5-a795-4ff3a7e311f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:35:32.712437 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f34449-3d70-4da5-a795-4ff3a7e311f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:35:32.712639 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f34449-3d70-4da5-a795-4ff3a7e311f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:35:32.713455 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f34449-3d70-4da5-a795-4ff3a7e311f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:35:32.714212 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f34449-3d70-4da5-a795-4ff3a7e311f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:35:32.716619 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f34449-3d70-4da5-a795-4ff3a7e311f7 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:35:32.735666 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a6b1a-4c29-490a-a2b1-445a3b041ac0 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:35:32.740801 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1341/2642] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:35:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:35:32.747454 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a6b1a-4c29-490a-a2b1-445a3b041ac0 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:35:32.753178 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c45cb5-db3c-431b-be75-17ae2a1df392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:35:32.753497 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c45cb5-db3c-431b-be75-17ae2a1df392 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:35:32.754404 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c45cb5-db3c-431b-be75-17ae2a1df392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:35:32.754404 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c45cb5-db3c-431b-be75-17ae2a1df392 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:35:32.755037 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c45cb5-db3c-431b-be75-17ae2a1df392 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:35:32.757097 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c45cb5-db3c-431b-be75-17ae2a1df392 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:35:32.867648 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1302/2643] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:35:32 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:35:32.869073 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f34449-3d70-4da5-a795-4ff3a7e311f7 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:35:32.880916 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f34449-3d70-4da5-a795-4ff3a7e311f7 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:35:32.901233 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:35:32.950327 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9556f7-d414-4013-ab57-43c0e819ebfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:35:32.951104 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9556f7-d414-4013-ab57-43c0e819ebfe admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:35:32.951464 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9556f7-d414-4013-ab57-43c0e819ebfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:35:32.952591 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9556f7-d414-4013-ab57-43c0e819ebfe admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:35:32.955417 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9556f7-d414-4013-ab57-43c0e819ebfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:35:32.955417 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9556f7-d414-4013-ab57-43c0e819ebfe admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:35:33.075077 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1342/2644] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:35:32 2022] POST /identity/v3/users => generated 293 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:35:33.080312 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c45cb5-db3c-431b-be75-17ae2a1df392 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:35:33.091285 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c45cb5-db3c-431b-be75-17ae2a1df392 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:35:33.098662 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f390f818-7622-4c64-8393-5325ba8e7558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:35:33.099366 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390f818-7622-4c64-8393-5325ba8e7558 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:35:33.099715 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390f818-7622-4c64-8393-5325ba8e7558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:35:33.100052 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390f818-7622-4c64-8393-5325ba8e7558 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:35:33.100985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f390f818-7622-4c64-8393-5325ba8e7558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:35:33.103898 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f390f818-7622-4c64-8393-5325ba8e7558 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:35:33.117624 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1303/2645] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:35:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:35:33.119727 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9556f7-d414-4013-ab57-43c0e819ebfe admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:35:33.133112 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9556f7-d414-4013-ab57-43c0e819ebfe admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:35:33.141185 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c204574-1134-44d3-bf75-6f408cbd5676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:35:33.141947 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c204574-1134-44d3-bf75-6f408cbd5676 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:35:33.142256 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c204574-1134-44d3-bf75-6f408cbd5676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:35:33.142525 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c204574-1134-44d3-bf75-6f408cbd5676 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:35:33.143259 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c204574-1134-44d3-bf75-6f408cbd5676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:35:33.145065 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c204574-1134-44d3-bf75-6f408cbd5676 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:35:33.156839 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1343/2646] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:35:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:35:33.161258 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f390f818-7622-4c64-8393-5325ba8e7558 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:35:33.171456 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f390f818-7622-4c64-8393-5325ba8e7558 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:35:33.176097 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88dcdeb-9442-41ab-bd2e-2ab4f4d96349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:35:33.176832 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88dcdeb-9442-41ab-bd2e-2ab4f4d96349 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:35:33.177114 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88dcdeb-9442-41ab-bd2e-2ab4f4d96349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:35:33.177389 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88dcdeb-9442-41ab-bd2e-2ab4f4d96349 admin admin] PATH_INFO: `/v3/projects/bc63bf4a15624098aa5568cad2c003ae/users/0cb8261b328c4cd8956762971fefceee/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:35:33.201075 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88dcdeb-9442-41ab-bd2e-2ab4f4d96349 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc63bf4a15624098aa5568cad2c003ae, user_id=0cb8261b328c4cd8956762971fefceee, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:35:33.204461 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88dcdeb-9442-41ab-bd2e-2ab4f4d96349 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:35:33.284070 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1304/2647] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:35:33 2022] PUT /identity/v3/projects/bc63bf4a15624098aa5568cad2c003ae/users/0cb8261b328c4cd8956762971fefceee/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:35:33.289257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400186e4-787c-4834-a33c-3418f0e6cbba None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:35:33.289537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400186e4-787c-4834-a33c-3418f0e6cbba None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:35:33.289810 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400186e4-787c-4834-a33c-3418f0e6cbba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:35:33.308297 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-400186e4-787c-4834-a33c-3418f0e6cbba None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:35:33.309510 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-400186e4-787c-4834-a33c-3418f0e6cbba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:35:33.366755 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1344/2648] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:35:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:35:33.377256 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88dcdeb-9442-41ab-bd2e-2ab4f4d96349 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:35:33.378944 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f88dcdeb-9442-41ab-bd2e-2ab4f4d96349 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:35:33.423226 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:35:33.437044 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88dcdeb-9442-41ab-bd2e-2ab4f4d96349 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:35:33.440717 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9f047-3daf-45ce-8ae7-bd438cf845e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:35:33.441322 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9f047-3daf-45ce-8ae7-bd438cf845e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:35:33.441599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9f047-3daf-45ce-8ae7-bd438cf845e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:35:33.441866 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9f047-3daf-45ce-8ae7-bd438cf845e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:35:33.451254 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b9f047-3daf-45ce-8ae7-bd438cf845e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:35:33.452728 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b9f047-3daf-45ce-8ae7-bd438cf845e0 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:35:33.461956 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1305/2649] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:35:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:35:38.556532 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1525aa6-51d7-46fe-bb71-66e1e154c3aa None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:35:38.557088 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1525aa6-51d7-46fe-bb71-66e1e154c3aa None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:35:38.557275 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1525aa6-51d7-46fe-bb71-66e1e154c3aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:35:38.588713 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-d1525aa6-51d7-46fe-bb71-66e1e154c3aa None None] MFA Rules not processed for user `0cb8261b328c4cd8956762971fefceee`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:35:38.590128 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-d1525aa6-51d7-46fe-bb71-66e1e154c3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:35:38.682318 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1345/2650] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:35:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:35:38.696821 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9f047-3daf-45ce-8ae7-bd438cf845e0 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:35:38.698312 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-57b9f047-3daf-45ce-8ae7-bd438cf845e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:35:38.790297 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9f047-3daf-45ce-8ae7-bd438cf845e0 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:35:38.796776 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46284729-f1b6-400d-99de-04cf756cb9fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:35:38.797471 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46284729-f1b6-400d-99de-04cf756cb9fd service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:35:38.797746 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46284729-f1b6-400d-99de-04cf756cb9fd service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:35:38.798166 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46284729-f1b6-400d-99de-04cf756cb9fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:35:38.806600 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46284729-f1b6-400d-99de-04cf756cb9fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:35:38.808250 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46284729-f1b6-400d-99de-04cf756cb9fd service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:35:38.819703 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1306/2651] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:35:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:35:41.086447 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15deef9d-1f26-417d-b912-829d056425f8 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:35:41.088772 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15deef9d-1f26-417d-b912-829d056425f8 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:35:41.088772 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15deef9d-1f26-417d-b912-829d056425f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:35:41.107417 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-15deef9d-1f26-417d-b912-829d056425f8 None None] MFA Rules not processed for user `0cb8261b328c4cd8956762971fefceee`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:35:41.108775 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-15deef9d-1f26-417d-b912-829d056425f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:35:41.116078 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1346/2652] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:35:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:35:41.450359 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46284729-f1b6-400d-99de-04cf756cb9fd service nova] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:35:41.461560 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46284729-f1b6-400d-99de-04cf756cb9fd service nova] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:35:41.465306 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e690f804-426f-4ced-a18e-4b2b7e32584a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:35:41.465923 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690f804-426f-4ced-a18e-4b2b7e32584a service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:35:41.466195 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690f804-426f-4ced-a18e-4b2b7e32584a service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:35:41.466453 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690f804-426f-4ced-a18e-4b2b7e32584a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:35:41.469095 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e690f804-426f-4ced-a18e-4b2b7e32584a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:35:41.479629 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e690f804-426f-4ced-a18e-4b2b7e32584a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:35:41.481267 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e690f804-426f-4ced-a18e-4b2b7e32584a service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:35:41.491262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1307/2653] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:35:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:35:41.930437 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15deef9d-1f26-417d-b912-829d056425f8 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:35:41.931848 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-15deef9d-1f26-417d-b912-829d056425f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:35:41.965468 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:35:42.015971 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15deef9d-1f26-417d-b912-829d056425f8 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:35:42.021618 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d42c98-30c0-47fe-911e-b5abf3fcee1a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff761f6f81a46098f94bd2ec2be9c1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:35:42.022616 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d42c98-30c0-47fe-911e-b5abf3fcee1a service placement] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:35:42.022900 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d42c98-30c0-47fe-911e-b5abf3fcee1a service placement] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:35:42.023176 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d42c98-30c0-47fe-911e-b5abf3fcee1a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:35:42.025090 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-38d42c98-30c0-47fe-911e-b5abf3fcee1a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:35:42.038355 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d42c98-30c0-47fe-911e-b5abf3fcee1a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:35:42.040122 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d42c98-30c0-47fe-911e-b5abf3fcee1a service placement] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:35:42.051083 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1347/2654] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:35:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:35:45.071972 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e690f804-426f-4ced-a18e-4b2b7e32584a service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:35:45.073598 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e690f804-426f-4ced-a18e-4b2b7e32584a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:35:45.085299 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e690f804-426f-4ced-a18e-4b2b7e32584a service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:35:45.090935 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa5c878-ea92-482e-be22-77d4e0160e4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:35:45.091548 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5c878-ea92-482e-be22-77d4e0160e4a service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:35:45.091822 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5c878-ea92-482e-be22-77d4e0160e4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:35:45.092089 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5c878-ea92-482e-be22-77d4e0160e4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:35:45.094427 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-9aa5c878-ea92-482e-be22-77d4e0160e4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:35:45.105670 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa5c878-ea92-482e-be22-77d4e0160e4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:35:45.107091 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa5c878-ea92-482e-be22-77d4e0160e4a service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:35:45.115390 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1308/2655] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:35:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:35:52.046554 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=22 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:35:55.111638 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=54 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:35:57.223485 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d42c98-30c0-47fe-911e-b5abf3fcee1a service placement] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:35:57.225191 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-38d42c98-30c0-47fe-911e-b5abf3fcee1a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:35:57.241777 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d42c98-30c0-47fe-911e-b5abf3fcee1a service placement] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:35:57.247228 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952612b3-beb9-493a-ade9-5bcb9aaad704 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:35:57.247856 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952612b3-beb9-493a-ade9-5bcb9aaad704 service nova] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:35:57.248133 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952612b3-beb9-493a-ade9-5bcb9aaad704 service nova] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:35:57.248402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952612b3-beb9-493a-ade9-5bcb9aaad704 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:35:57.250215 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-952612b3-beb9-493a-ade9-5bcb9aaad704 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:35:57.263920 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952612b3-beb9-493a-ade9-5bcb9aaad704 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:35:57.265535 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952612b3-beb9-493a-ade9-5bcb9aaad704 service nova] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:35:57.277454 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1348/2656] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:35:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:36:06.190833 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa5c878-ea92-482e-be22-77d4e0160e4a service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:36:06.205493 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa5c878-ea92-482e-be22-77d4e0160e4a service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:36:06.211151 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7141717-4338-4610-b1f8-44c214c104ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:36:06.211871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7141717-4338-4610-b1f8-44c214c104ef service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:36:06.212217 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7141717-4338-4610-b1f8-44c214c104ef service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:36:06.212630 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7141717-4338-4610-b1f8-44c214c104ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:36:06.214491 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c7141717-4338-4610-b1f8-44c214c104ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:36:06.230272 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7141717-4338-4610-b1f8-44c214c104ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:36:06.230866 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7141717-4338-4610-b1f8-44c214c104ef service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:36:06.243329 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1309/2657] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:36:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:36:07.272567 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=6 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:36:11.648744 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952612b3-beb9-493a-ade9-5bcb9aaad704 service nova] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:36:11.658201 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952612b3-beb9-493a-ade9-5bcb9aaad704 service nova] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:36:11.661887 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0e296-533f-4fbd-a440-4c0ad0b4823c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:36:11.662525 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0e296-533f-4fbd-a440-4c0ad0b4823c service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:36:11.662803 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0e296-533f-4fbd-a440-4c0ad0b4823c service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:36:11.663077 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0e296-533f-4fbd-a440-4c0ad0b4823c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:36:11.664857 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-19c0e296-533f-4fbd-a440-4c0ad0b4823c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:36:11.677218 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c0e296-533f-4fbd-a440-4c0ad0b4823c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:36:11.678639 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c0e296-533f-4fbd-a440-4c0ad0b4823c service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:36:11.688128 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1349/2658] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:36:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:36:15.172076 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7141717-4338-4610-b1f8-44c214c104ef service nova] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:36:15.173625 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c7141717-4338-4610-b1f8-44c214c104ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:36:15.188392 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7141717-4338-4610-b1f8-44c214c104ef service nova] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:36:15.193735 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f6ca6-883d-4051-9e2a-402b754966c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:36:15.194373 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f6ca6-883d-4051-9e2a-402b754966c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:36:15.194647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f6ca6-883d-4051-9e2a-402b754966c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:36:15.194919 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f6ca6-883d-4051-9e2a-402b754966c3 admin admin] PATH_INFO: `/v3/users/084b18670c2a4c1396d5912d130b2bac` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:36:15.208996 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132f6ca6-883d-4051-9e2a-402b754966c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=084b18670c2a4c1396d5912d130b2bac)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:36:15.210897 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132f6ca6-883d-4051-9e2a-402b754966c3 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:36:15.342460 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-132f6ca6-883d-4051-9e2a-402b754966c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '084b18670c2a4c1396d5912d130b2bac'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:36:15.354539 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-132f6ca6-883d-4051-9e2a-402b754966c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '084b18670c2a4c1396d5912d130b2bac'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:36:15.409818 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-132f6ca6-883d-4051-9e2a-402b754966c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '084b18670c2a4c1396d5912d130b2bac'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:36:15.410461 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-132f6ca6-883d-4051-9e2a-402b754966c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '084b18670c2a4c1396d5912d130b2bac'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:36:15.417514 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-132f6ca6-883d-4051-9e2a-402b754966c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '084b18670c2a4c1396d5912d130b2bac'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:36:15.423429 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1310/2659] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:36:15 2022] DELETE /identity/v3/users/084b18670c2a4c1396d5912d130b2bac => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:36:15.668232 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0e296-533f-4fbd-a440-4c0ad0b4823c service neutron] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:36:15.680322 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0e296-533f-4fbd-a440-4c0ad0b4823c service neutron] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:36:15.743596 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209dd01b-694d-4317-a78e-85402516cd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:36:15.743596 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209dd01b-694d-4317-a78e-85402516cd2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:36:15.743596 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209dd01b-694d-4317-a78e-85402516cd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:36:15.743596 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209dd01b-694d-4317-a78e-85402516cd2a admin admin] PATH_INFO: `/v3/projects/5c13896c64e44b60838d3f313407433f` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:36:15.751893 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209dd01b-694d-4317-a78e-85402516cd2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c13896c64e44b60838d3f313407433f)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:36:15.754202 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209dd01b-694d-4317-a78e-85402516cd2a admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:36:15.901607 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-209dd01b-694d-4317-a78e-85402516cd2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c13896c64e44b60838d3f313407433f', 'request_id': 'req-209dd01b-694d-4317-a78e-85402516cd2a'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:36:15.908088 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-209dd01b-694d-4317-a78e-85402516cd2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c13896c64e44b60838d3f313407433f', 'request_id': 'req-209dd01b-694d-4317-a78e-85402516cd2a'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:36:15.914207 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-209dd01b-694d-4317-a78e-85402516cd2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c13896c64e44b60838d3f313407433f', 'request_id': 'req-209dd01b-694d-4317-a78e-85402516cd2a'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:36:16.009256 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-209dd01b-694d-4317-a78e-85402516cd2a admin admin] The token cache is being invalidate because project 5c13896c64e44b60838d3f313407433f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:36:16.009471 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-209dd01b-694d-4317-a78e-85402516cd2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:36:16.010101 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-209dd01b-694d-4317-a78e-85402516cd2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:36:16.011622 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1350/2660] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:36:15 2022] DELETE /identity/v3/projects/5c13896c64e44b60838d3f313407433f => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:36:16.016820 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7f25e-604c-4a1a-a537-15ae8e1b7f0b None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:36:16.017111 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7f25e-604c-4a1a-a537-15ae8e1b7f0b None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:36:16.017388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7f25e-604c-4a1a-a537-15ae8e1b7f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:36:16.037950 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-55c7f25e-604c-4a1a-a537-15ae8e1b7f0b None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:36:16.039159 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-55c7f25e-604c-4a1a-a537-15ae8e1b7f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:36:16.098490 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1311/2661] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:36:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:36:16.107186 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f83f6f-315c-4602-9caa-f4f3712f9754 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:36:16.107466 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f83f6f-315c-4602-9caa-f4f3712f9754 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:36:16.107741 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f83f6f-315c-4602-9caa-f4f3712f9754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:36:16.126957 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-33f83f6f-315c-4602-9caa-f4f3712f9754 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:36:16.128165 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-33f83f6f-315c-4602-9caa-f4f3712f9754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:36:16.136339 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1351/2662] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:36:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:36:16.144137 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7f25e-604c-4a1a-a537-15ae8e1b7f0b None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:36:16.153740 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7f25e-604c-4a1a-a537-15ae8e1b7f0b None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:36:16.158342 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c9959-3458-4590-98ad-6e6e83905065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:36:16.158946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c9959-3458-4590-98ad-6e6e83905065 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:36:16.159340 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c9959-3458-4590-98ad-6e6e83905065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:36:16.159616 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c9959-3458-4590-98ad-6e6e83905065 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:36:16.161045 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9c9959-3458-4590-98ad-6e6e83905065 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:36:16.161347 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9c9959-3458-4590-98ad-6e6e83905065 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:36:16.162922 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9c9959-3458-4590-98ad-6e6e83905065 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:36:16.174311 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1312/2663] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:36:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:36:16.178500 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f83f6f-315c-4602-9caa-f4f3712f9754 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:36:16.187652 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f83f6f-315c-4602-9caa-f4f3712f9754 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:36:16.192350 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c2b60-a5de-48ee-91d9-c2757a14df5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:36:16.192976 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c2b60-a5de-48ee-91d9-c2757a14df5c admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:36:16.193255 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c2b60-a5de-48ee-91d9-c2757a14df5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:36:16.193521 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c2b60-a5de-48ee-91d9-c2757a14df5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:36:16.194180 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841c2b60-a5de-48ee-91d9-c2757a14df5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:36:16.196057 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841c2b60-a5de-48ee-91d9-c2757a14df5c admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:36:16.285886 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1352/2664] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:36:16 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:36:16.290096 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c9959-3458-4590-98ad-6e6e83905065 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:36:16.300041 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c9959-3458-4590-98ad-6e6e83905065 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:36:16.333460 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:36:16.356860 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa7bc55-a7dc-49bb-9a67-afae4be29d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:36:16.357472 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7bc55-a7dc-49bb-9a67-afae4be29d7e admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:36:16.357751 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7bc55-a7dc-49bb-9a67-afae4be29d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:36:16.358024 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7bc55-a7dc-49bb-9a67-afae4be29d7e admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:36:16.358674 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa7bc55-a7dc-49bb-9a67-afae4be29d7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:36:16.360596 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa7bc55-a7dc-49bb-9a67-afae4be29d7e admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:36:16.460066 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1313/2665] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:36:16 2022] POST /identity/v3/users => generated 287 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:36:16.464575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c2b60-a5de-48ee-91d9-c2757a14df5c admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:36:16.475061 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c2b60-a5de-48ee-91d9-c2757a14df5c admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:36:16.479720 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027230bd-982d-4289-bb28-6c68eef2c2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:36:16.481437 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027230bd-982d-4289-bb28-6c68eef2c2cd admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:36:16.481437 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027230bd-982d-4289-bb28-6c68eef2c2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:36:16.481437 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027230bd-982d-4289-bb28-6c68eef2c2cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:36:16.481738 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027230bd-982d-4289-bb28-6c68eef2c2cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:36:16.483101 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027230bd-982d-4289-bb28-6c68eef2c2cd admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:36:16.492834 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1353/2666] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:36:16.496773 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa7bc55-a7dc-49bb-9a67-afae4be29d7e admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:36:16.507447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa7bc55-a7dc-49bb-9a67-afae4be29d7e admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:36:16.512013 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b8d88-62bc-490f-9bf5-8b7b4f6aaa6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:36:16.512639 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b8d88-62bc-490f-9bf5-8b7b4f6aaa6d admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:36:16.512915 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b8d88-62bc-490f-9bf5-8b7b4f6aaa6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:36:16.513187 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b8d88-62bc-490f-9bf5-8b7b4f6aaa6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:36:16.513903 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b8d88-62bc-490f-9bf5-8b7b4f6aaa6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:36:16.515485 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b8d88-62bc-490f-9bf5-8b7b4f6aaa6d admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:36:16.526227 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1314/2667] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:36:16.530411 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027230bd-982d-4289-bb28-6c68eef2c2cd admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:36:16.540640 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027230bd-982d-4289-bb28-6c68eef2c2cd admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:36:16.545226 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67c98d-0770-4f68-a8be-584595c0a43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:36:16.545881 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67c98d-0770-4f68-a8be-584595c0a43f admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:36:16.546162 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67c98d-0770-4f68-a8be-584595c0a43f admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:36:16.546436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67c98d-0770-4f68-a8be-584595c0a43f admin admin] PATH_INFO: `/v3/projects/b01b3aa5e1464228ac9c4efc1a24a419/users/6fe31b19c15949daa1a6afe88538c2c1/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:36:16.561284 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67c98d-0770-4f68-a8be-584595c0a43f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b01b3aa5e1464228ac9c4efc1a24a419, user_id=6fe31b19c15949daa1a6afe88538c2c1, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:36:16.563951 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67c98d-0770-4f68-a8be-584595c0a43f admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:36:16.633681 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1354/2668] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:36:16 2022] PUT /identity/v3/projects/b01b3aa5e1464228ac9c4efc1a24a419/users/6fe31b19c15949daa1a6afe88538c2c1/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:36:16.644678 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba2d1a-6b75-45d7-bb2f-3dd6613d65b6 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:36:16.644969 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba2d1a-6b75-45d7-bb2f-3dd6613d65b6 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:36:16.645242 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba2d1a-6b75-45d7-bb2f-3dd6613d65b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:36:16.676531 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-ddba2d1a-6b75-45d7-bb2f-3dd6613d65b6 None None] MFA Rules not processed for user `6fe31b19c15949daa1a6afe88538c2c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:36:16.677757 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ddba2d1a-6b75-45d7-bb2f-3dd6613d65b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:36:16.745616 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1315/2669] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:36:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:36:16.757462 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67c98d-0770-4f68-a8be-584595c0a43f admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:36:16.759180 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-be67c98d-0770-4f68-a8be-584595c0a43f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:36:16.765620 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:36:16.830431 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67c98d-0770-4f68-a8be-584595c0a43f admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:36:16.836444 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715df58a-e39e-4a2a-a53b-d97416f02427 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:36:16.837051 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715df58a-e39e-4a2a-a53b-d97416f02427 service nova] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:36:16.837331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715df58a-e39e-4a2a-a53b-d97416f02427 service nova] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:36:16.837602 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715df58a-e39e-4a2a-a53b-d97416f02427 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:36:16.845462 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715df58a-e39e-4a2a-a53b-d97416f02427 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:36:16.847128 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715df58a-e39e-4a2a-a53b-d97416f02427 service nova] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:36:16.855770 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1355/2670] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:36:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:36:17.064737 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba2d1a-6b75-45d7-bb2f-3dd6613d65b6 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:36:17.127014 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba2d1a-6b75-45d7-bb2f-3dd6613d65b6 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:36:17.131993 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec0dfa-4604-41d4-bb33-175335381a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:36:17.132730 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec0dfa-4604-41d4-bb33-175335381a63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:36:17.133012 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec0dfa-4604-41d4-bb33-175335381a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:36:17.133328 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec0dfa-4604-41d4-bb33-175335381a63 admin admin] PATH_INFO: `/v3/users/6fe31b19c15949daa1a6afe88538c2c1` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:36:17.134503 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ec0dfa-4604-41d4-bb33-175335381a63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fe31b19c15949daa1a6afe88538c2c1)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:36:17.136475 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ec0dfa-4604-41d4-bb33-175335381a63 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:36:17.291214 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:36:17.292755 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-d0ec0dfa-4604-41d4-bb33-175335381a63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fe31b19c15949daa1a6afe88538c2c1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:36:17.305362 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-d0ec0dfa-4604-41d4-bb33-175335381a63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fe31b19c15949daa1a6afe88538c2c1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:36:17.380966 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-d0ec0dfa-4604-41d4-bb33-175335381a63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fe31b19c15949daa1a6afe88538c2c1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:36:17.381516 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-d0ec0dfa-4604-41d4-bb33-175335381a63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fe31b19c15949daa1a6afe88538c2c1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:36:17.388536 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-d0ec0dfa-4604-41d4-bb33-175335381a63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fe31b19c15949daa1a6afe88538c2c1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:36:17.394925 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1316/2671] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:36:17 2022] DELETE /identity/v3/users/6fe31b19c15949daa1a6afe88538c2c1 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:36:17.399418 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db9561-1950-4416-bcc1-d707539d1b16 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:36:17.399706 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db9561-1950-4416-bcc1-d707539d1b16 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:36:17.399979 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db9561-1950-4416-bcc1-d707539d1b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:36:17.419102 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-46db9561-1950-4416-bcc1-d707539d1b16 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:36:17.420307 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-46db9561-1950-4416-bcc1-d707539d1b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:36:17.477319 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1356/2672] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:36:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:36:17.487016 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec0dfa-4604-41d4-bb33-175335381a63 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:36:17.488470 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d0ec0dfa-4604-41d4-bb33-175335381a63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:36:17.546845 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec0dfa-4604-41d4-bb33-175335381a63 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:36:17.551146 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c0b9bd-9bb2-42a7-8c6d-1f01f87cae25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:36:17.551758 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0b9bd-9bb2-42a7-8c6d-1f01f87cae25 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:36:17.552034 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0b9bd-9bb2-42a7-8c6d-1f01f87cae25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:36:17.552303 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0b9bd-9bb2-42a7-8c6d-1f01f87cae25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:36:17.561231 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c0b9bd-9bb2-42a7-8c6d-1f01f87cae25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:36:17.562654 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c0b9bd-9bb2-42a7-8c6d-1f01f87cae25 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:36:17.571924 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1317/2673] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:36:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:36:17.970414 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46db9561-1950-4416-bcc1-d707539d1b16 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:36:17.972398 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-46db9561-1950-4416-bcc1-d707539d1b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:36:17.985870 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46db9561-1950-4416-bcc1-d707539d1b16 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:36:17.991239 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0398374b-d20a-482e-834c-a3ce63a98256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:36:17.992073 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398374b-d20a-482e-834c-a3ce63a98256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:36:17.992524 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398374b-d20a-482e-834c-a3ce63a98256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:36:17.992702 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398374b-d20a-482e-834c-a3ce63a98256 admin admin] PATH_INFO: `/v3/projects/b01b3aa5e1464228ac9c4efc1a24a419` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:36:17.993955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0398374b-d20a-482e-834c-a3ce63a98256 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b01b3aa5e1464228ac9c4efc1a24a419)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:36:17.995920 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0398374b-d20a-482e-834c-a3ce63a98256 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:36:18.125509 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-0398374b-d20a-482e-834c-a3ce63a98256 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b01b3aa5e1464228ac9c4efc1a24a419', 'request_id': 'req-0398374b-d20a-482e-834c-a3ce63a98256'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:36:18.132723 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-0398374b-d20a-482e-834c-a3ce63a98256 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b01b3aa5e1464228ac9c4efc1a24a419', 'request_id': 'req-0398374b-d20a-482e-834c-a3ce63a98256'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:36:18.138623 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-0398374b-d20a-482e-834c-a3ce63a98256 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b01b3aa5e1464228ac9c4efc1a24a419', 'request_id': 'req-0398374b-d20a-482e-834c-a3ce63a98256'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:36:18.214192 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-0398374b-d20a-482e-834c-a3ce63a98256 admin admin] The token cache is being invalidate because project b01b3aa5e1464228ac9c4efc1a24a419 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:36:18.214587 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-0398374b-d20a-482e-834c-a3ce63a98256 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:36:18.215901 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-0398374b-d20a-482e-834c-a3ce63a98256 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:36:18.218357 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1357/2674] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:36:17 2022] DELETE /identity/v3/projects/b01b3aa5e1464228ac9c4efc1a24a419 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:36:18.223931 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61e24e-1d1d-428a-9d5c-add47b30afcf None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:36:18.224339 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61e24e-1d1d-428a-9d5c-add47b30afcf None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:36:18.224857 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61e24e-1d1d-428a-9d5c-add47b30afcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:36:18.247422 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-1e61e24e-1d1d-428a-9d5c-add47b30afcf None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:36:18.248862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-1e61e24e-1d1d-428a-9d5c-add47b30afcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:36:18.319217 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1318/2675] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:36:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:36:18.327571 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478992b0-34ed-4a60-94ba-28c6484dcb31 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:36:18.327850 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478992b0-34ed-4a60-94ba-28c6484dcb31 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:36:18.328122 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478992b0-34ed-4a60-94ba-28c6484dcb31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:36:18.348475 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-478992b0-34ed-4a60-94ba-28c6484dcb31 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:36:18.349677 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-478992b0-34ed-4a60-94ba-28c6484dcb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:36:18.357687 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1358/2676] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:36:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:36:18.362074 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61e24e-1d1d-428a-9d5c-add47b30afcf None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:36:18.374234 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61e24e-1d1d-428a-9d5c-add47b30afcf None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:36:18.381062 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348317c-487b-49f0-b298-fa815e2de630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:36:18.381943 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348317c-487b-49f0-b298-fa815e2de630 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:36:18.382382 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348317c-487b-49f0-b298-fa815e2de630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:36:18.382717 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348317c-487b-49f0-b298-fa815e2de630 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:36:18.383815 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d348317c-487b-49f0-b298-fa815e2de630 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:36:18.384380 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d348317c-487b-49f0-b298-fa815e2de630 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:36:18.386920 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d348317c-487b-49f0-b298-fa815e2de630 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:36:18.400158 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1319/2677] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:36:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:36:18.405229 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478992b0-34ed-4a60-94ba-28c6484dcb31 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:36:18.414548 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478992b0-34ed-4a60-94ba-28c6484dcb31 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:36:18.419855 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98dc892-fc66-453e-a8bd-f9f6b91860ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:36:18.420762 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98dc892-fc66-453e-a8bd-f9f6b91860ca admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:36:18.421293 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98dc892-fc66-453e-a8bd-f9f6b91860ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:36:18.421561 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98dc892-fc66-453e-a8bd-f9f6b91860ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:36:18.422225 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98dc892-fc66-453e-a8bd-f9f6b91860ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:36:18.424114 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98dc892-fc66-453e-a8bd-f9f6b91860ca admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:36:18.429304 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:36:18.537028 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1359/2678] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:36:18 2022] POST /identity/v3/projects => generated 377 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:36:18.541085 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348317c-487b-49f0-b298-fa815e2de630 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:36:18.551643 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348317c-487b-49f0-b298-fa815e2de630 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:36:18.598113 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:36:18.606733 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53ce8f-2988-4ec6-9f3c-ae5ed2d9ebbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:36:18.607348 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53ce8f-2988-4ec6-9f3c-ae5ed2d9ebbd admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:36:18.607625 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53ce8f-2988-4ec6-9f3c-ae5ed2d9ebbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:36:18.607899 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53ce8f-2988-4ec6-9f3c-ae5ed2d9ebbd admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:36:18.608584 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c53ce8f-2988-4ec6-9f3c-ae5ed2d9ebbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:36:18.610485 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c53ce8f-2988-4ec6-9f3c-ae5ed2d9ebbd admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:36:18.695595 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1320/2679] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:36:18 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:36:18.699863 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98dc892-fc66-453e-a8bd-f9f6b91860ca admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:36:18.710231 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98dc892-fc66-453e-a8bd-f9f6b91860ca admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:36:18.714951 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e5af8c-ede7-44c8-b421-a8d279b5b45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:36:18.715560 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5af8c-ede7-44c8-b421-a8d279b5b45d admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:36:18.715842 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5af8c-ede7-44c8-b421-a8d279b5b45d admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:36:18.716116 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5af8c-ede7-44c8-b421-a8d279b5b45d admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:36:18.716876 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e5af8c-ede7-44c8-b421-a8d279b5b45d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:36:18.718500 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e5af8c-ede7-44c8-b421-a8d279b5b45d admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:36:18.729338 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1360/2680] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:36:18.733306 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53ce8f-2988-4ec6-9f3c-ae5ed2d9ebbd admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:36:18.744054 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53ce8f-2988-4ec6-9f3c-ae5ed2d9ebbd admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:36:18.748679 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21acdb5-4ae9-4235-a379-ffd8eb51608b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:36:18.754811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21acdb5-4ae9-4235-a379-ffd8eb51608b admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:36:18.754811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21acdb5-4ae9-4235-a379-ffd8eb51608b admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:36:18.754811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21acdb5-4ae9-4235-a379-ffd8eb51608b admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:36:18.754811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21acdb5-4ae9-4235-a379-ffd8eb51608b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:36:18.754811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21acdb5-4ae9-4235-a379-ffd8eb51608b admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:36:18.762540 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1321/2681] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:36:18.766823 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e5af8c-ede7-44c8-b421-a8d279b5b45d admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:36:18.777149 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e5af8c-ede7-44c8-b421-a8d279b5b45d admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:36:18.781856 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8efc2f-d6ba-4259-9494-a01e1fbb63b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:36:18.783541 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8efc2f-d6ba-4259-9494-a01e1fbb63b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:36:18.783541 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8efc2f-d6ba-4259-9494-a01e1fbb63b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:36:18.783541 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8efc2f-d6ba-4259-9494-a01e1fbb63b1 admin admin] PATH_INFO: `/v3/projects/aba0546fe49b48b8af71333bfc63aaaa/users/5314a782e5d741f68e541300751f05a7/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:36:18.798726 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8efc2f-d6ba-4259-9494-a01e1fbb63b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aba0546fe49b48b8af71333bfc63aaaa, user_id=5314a782e5d741f68e541300751f05a7, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:36:18.802303 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8efc2f-d6ba-4259-9494-a01e1fbb63b1 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:36:18.918781 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1361/2682] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:36:18 2022] PUT /identity/v3/projects/aba0546fe49b48b8af71333bfc63aaaa/users/5314a782e5d741f68e541300751f05a7/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:36:18.929402 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30e33c-fa5f-43f4-a706-5e72233dd27a None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:36:18.929682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30e33c-fa5f-43f4-a706-5e72233dd27a None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:36:18.932518 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30e33c-fa5f-43f4-a706-5e72233dd27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:36:18.957733 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-5e30e33c-fa5f-43f4-a706-5e72233dd27a None None] MFA Rules not processed for user `5314a782e5d741f68e541300751f05a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:36:18.958935 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5e30e33c-fa5f-43f4-a706-5e72233dd27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:36:19.022362 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1322/2683] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:36:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:36:19.026794 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8efc2f-d6ba-4259-9494-a01e1fbb63b1 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:36:19.094393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8efc2f-d6ba-4259-9494-a01e1fbb63b1 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:36:19.099681 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f7a471-d3ee-4a1a-8718-bce0116cfeb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:36:19.100267 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f7a471-d3ee-4a1a-8718-bce0116cfeb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:36:19.100573 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f7a471-d3ee-4a1a-8718-bce0116cfeb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:36:19.100844 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f7a471-d3ee-4a1a-8718-bce0116cfeb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:36:19.101505 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f7a471-d3ee-4a1a-8718-bce0116cfeb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:36:19.103387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f7a471-d3ee-4a1a-8718-bce0116cfeb1 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:36:19.182574 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1362/2684] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:36:19 2022] POST /identity/v3/projects => generated 375 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:36:19.186953 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e30e33c-fa5f-43f4-a706-5e72233dd27a None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:36:19.197807 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e30e33c-fa5f-43f4-a706-5e72233dd27a None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:36:19.252893 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc956c62-66af-4d24-a363-4bb184622f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:36:19.253485 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc956c62-66af-4d24-a363-4bb184622f01 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:36:19.253759 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc956c62-66af-4d24-a363-4bb184622f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:36:19.254028 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc956c62-66af-4d24-a363-4bb184622f01 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:36:19.254674 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc956c62-66af-4d24-a363-4bb184622f01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:36:19.256579 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc956c62-66af-4d24-a363-4bb184622f01 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:36:19.322998 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1323/2685] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:36:19 2022] POST /identity/v3/users => generated 294 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:36:19.326870 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f7a471-d3ee-4a1a-8718-bce0116cfeb1 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:36:19.337662 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f7a471-d3ee-4a1a-8718-bce0116cfeb1 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:36:19.342261 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a8a3b-69ba-4182-9dd5-3830c3e42f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:36:19.342857 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a8a3b-69ba-4182-9dd5-3830c3e42f90 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:36:19.343132 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a8a3b-69ba-4182-9dd5-3830c3e42f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:36:19.343400 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a8a3b-69ba-4182-9dd5-3830c3e42f90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:36:19.344441 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a8a3b-69ba-4182-9dd5-3830c3e42f90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:36:19.346017 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a8a3b-69ba-4182-9dd5-3830c3e42f90 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:36:19.356507 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1363/2686] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:36:19.360638 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc956c62-66af-4d24-a363-4bb184622f01 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:36:19.372323 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc956c62-66af-4d24-a363-4bb184622f01 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:36:19.377250 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e8cef-1c17-4a25-aa2e-6701ee79ab12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:36:19.378446 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e8cef-1c17-4a25-aa2e-6701ee79ab12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:36:19.378724 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e8cef-1c17-4a25-aa2e-6701ee79ab12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:36:19.379110 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e8cef-1c17-4a25-aa2e-6701ee79ab12 admin admin] PATH_INFO: `/v3/projects/1bbe979a71c94f32a84b6ccba3fed5b8/users/6095d0b6c0494984bafe8799ed083478/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:36:19.394614 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98e8cef-1c17-4a25-aa2e-6701ee79ab12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bbe979a71c94f32a84b6ccba3fed5b8, user_id=6095d0b6c0494984bafe8799ed083478, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:36:19.397307 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98e8cef-1c17-4a25-aa2e-6701ee79ab12 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:36:19.472310 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1324/2687] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:36:19 2022] PUT /identity/v3/projects/1bbe979a71c94f32a84b6ccba3fed5b8/users/6095d0b6c0494984bafe8799ed083478/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:36:19.476366 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a8a3b-69ba-4182-9dd5-3830c3e42f90 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:36:19.486214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a8a3b-69ba-4182-9dd5-3830c3e42f90 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:36:19.542522 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc23f0cc-6f90-435c-9408-653398f99a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:36:19.543136 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23f0cc-6f90-435c-9408-653398f99a49 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:36:19.543410 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23f0cc-6f90-435c-9408-653398f99a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:36:19.543676 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23f0cc-6f90-435c-9408-653398f99a49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:36:19.544359 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc23f0cc-6f90-435c-9408-653398f99a49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:36:19.545976 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc23f0cc-6f90-435c-9408-653398f99a49 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:36:19.555664 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1364/2688] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:36:19.559830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e8cef-1c17-4a25-aa2e-6701ee79ab12 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:36:19.570666 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e8cef-1c17-4a25-aa2e-6701ee79ab12 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:36:19.575394 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538a5139-4d7a-4085-8fac-fad42308d1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:36:19.575987 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a5139-4d7a-4085-8fac-fad42308d1b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:36:19.576269 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a5139-4d7a-4085-8fac-fad42308d1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:36:19.576585 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a5139-4d7a-4085-8fac-fad42308d1b8 admin admin] PATH_INFO: `/v3/projects/1bbe979a71c94f32a84b6ccba3fed5b8/users/6095d0b6c0494984bafe8799ed083478/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:36:19.578546 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538a5139-4d7a-4085-8fac-fad42308d1b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bbe979a71c94f32a84b6ccba3fed5b8, user_id=6095d0b6c0494984bafe8799ed083478, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:36:19.581903 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538a5139-4d7a-4085-8fac-fad42308d1b8 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:36:19.591910 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1325/2689] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:36:19 2022] PUT /identity/v3/projects/1bbe979a71c94f32a84b6ccba3fed5b8/users/6095d0b6c0494984bafe8799ed083478/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:36:19.604032 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ecaec-8880-4dff-b329-da479df48a5b None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:36:19.604316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ecaec-8880-4dff-b329-da479df48a5b None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:36:19.604624 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ecaec-8880-4dff-b329-da479df48a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:36:19.631931 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-446ecaec-8880-4dff-b329-da479df48a5b None None] MFA Rules not processed for user `6095d0b6c0494984bafe8799ed083478`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:36:19.633186 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-446ecaec-8880-4dff-b329-da479df48a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:36:19.641124 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:36:19.700582 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1365/2690] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:36:19 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:36:19.711358 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538a5139-4d7a-4085-8fac-fad42308d1b8 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:36:19.713069 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-538a5139-4d7a-4085-8fac-fad42308d1b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:36:19.727932 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:36:19.770556 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538a5139-4d7a-4085-8fac-fad42308d1b8 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:36:19.774378 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f604a3-ef51-47e0-94c6-401540774e47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:36:19.774984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f604a3-ef51-47e0-94c6-401540774e47 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:36:19.775258 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f604a3-ef51-47e0-94c6-401540774e47 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:36:19.775526 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f604a3-ef51-47e0-94c6-401540774e47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:36:19.831675 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f604a3-ef51-47e0-94c6-401540774e47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:36:19.833093 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f604a3-ef51-47e0-94c6-401540774e47 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:36:19.841904 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1326/2691] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:36:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:36:28.113470 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446ecaec-8880-4dff-b329-da479df48a5b None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:36:28.114870 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-446ecaec-8880-4dff-b329-da479df48a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:36:28.126413 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446ecaec-8880-4dff-b329-da479df48a5b None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:36:28.130893 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba32b04-5e53-4b37-aae7-484d59a11a49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:36:28.131491 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba32b04-5e53-4b37-aae7-484d59a11a49 service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:36:28.131767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba32b04-5e53-4b37-aae7-484d59a11a49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:36:28.132036 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba32b04-5e53-4b37-aae7-484d59a11a49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:36:28.140825 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba32b04-5e53-4b37-aae7-484d59a11a49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:36:28.142221 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba32b04-5e53-4b37-aae7-484d59a11a49 service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:36:28.151909 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1366/2692] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:36:28 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:36:29.837836 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=42 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:36:38.147801 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=31 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:37:00.264008 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f604a3-ef51-47e0-94c6-401540774e47 service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:00.326330 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f604a3-ef51-47e0-94c6-401540774e47 service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:00.330925 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f50a56-978f-4fa7-b749-fb118bd256cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:00.331555 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f50a56-978f-4fa7-b749-fb118bd256cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:00.331830 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f50a56-978f-4fa7-b749-fb118bd256cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:00.332101 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f50a56-978f-4fa7-b749-fb118bd256cb admin admin] PATH_INFO: `/v3/users/5314a782e5d741f68e541300751f05a7` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:00.333286 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f50a56-978f-4fa7-b749-fb118bd256cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=5314a782e5d741f68e541300751f05a7)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:00.335202 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f50a56-978f-4fa7-b749-fb118bd256cb admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:00.497124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-98f50a56-978f-4fa7-b749-fb118bd256cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5314a782e5d741f68e541300751f05a7'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:00.508995 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-98f50a56-978f-4fa7-b749-fb118bd256cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5314a782e5d741f68e541300751f05a7'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:00.597788 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-98f50a56-978f-4fa7-b749-fb118bd256cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5314a782e5d741f68e541300751f05a7'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:00.598403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-98f50a56-978f-4fa7-b749-fb118bd256cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5314a782e5d741f68e541300751f05a7'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:00.606557 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-98f50a56-978f-4fa7-b749-fb118bd256cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5314a782e5d741f68e541300751f05a7'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:00.612372 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1327/2693] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:37:00 2022] DELETE /identity/v3/users/5314a782e5d741f68e541300751f05a7 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:37:00.617599 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e8fd0-35d0-44af-bfa2-c47ce8863eda None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:00.618619 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e8fd0-35d0-44af-bfa2-c47ce8863eda None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:00.618619 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e8fd0-35d0-44af-bfa2-c47ce8863eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:00.637046 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-d36e8fd0-35d0-44af-bfa2-c47ce8863eda None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:37:00.638252 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-d36e8fd0-35d0-44af-bfa2-c47ce8863eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:37:00.695627 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1367/2694] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:37:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:37:00.705303 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f50a56-978f-4fa7-b749-fb118bd256cb admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:00.706678 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-98f50a56-978f-4fa7-b749-fb118bd256cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:37:00.762437 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f50a56-978f-4fa7-b749-fb118bd256cb admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:00.765910 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a0f606-8e97-4c61-924e-727bd5aa6fd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:00.766504 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0f606-8e97-4c61-924e-727bd5aa6fd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:00.766780 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0f606-8e97-4c61-924e-727bd5aa6fd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:00.767078 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0f606-8e97-4c61-924e-727bd5aa6fd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:00.776406 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a0f606-8e97-4c61-924e-727bd5aa6fd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:00.777836 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a0f606-8e97-4c61-924e-727bd5aa6fd5 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:00.786985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1328/2695] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:37:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:37:01.289425 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36e8fd0-35d0-44af-bfa2-c47ce8863eda None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:01.290869 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-d36e8fd0-35d0-44af-bfa2-c47ce8863eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:37:01.309286 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36e8fd0-35d0-44af-bfa2-c47ce8863eda None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:01.309286 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28f306-0b15-45c8-9cf9-d171bf897165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:01.309286 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28f306-0b15-45c8-9cf9-d171bf897165 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:01.309286 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28f306-0b15-45c8-9cf9-d171bf897165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:01.309619 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28f306-0b15-45c8-9cf9-d171bf897165 admin admin] PATH_INFO: `/v3/projects/aba0546fe49b48b8af71333bfc63aaaa` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:01.310516 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a28f306-0b15-45c8-9cf9-d171bf897165 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aba0546fe49b48b8af71333bfc63aaaa)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:01.312437 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a28f306-0b15-45c8-9cf9-d171bf897165 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:01.430580 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-0a28f306-0b15-45c8-9cf9-d171bf897165 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aba0546fe49b48b8af71333bfc63aaaa', 'request_id': 'req-0a28f306-0b15-45c8-9cf9-d171bf897165'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:01.436620 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-0a28f306-0b15-45c8-9cf9-d171bf897165 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aba0546fe49b48b8af71333bfc63aaaa', 'request_id': 'req-0a28f306-0b15-45c8-9cf9-d171bf897165'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:01.441942 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-0a28f306-0b15-45c8-9cf9-d171bf897165 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aba0546fe49b48b8af71333bfc63aaaa', 'request_id': 'req-0a28f306-0b15-45c8-9cf9-d171bf897165'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:01.533312 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-0a28f306-0b15-45c8-9cf9-d171bf897165 admin admin] The token cache is being invalidate because project aba0546fe49b48b8af71333bfc63aaaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:37:01.533599 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-0a28f306-0b15-45c8-9cf9-d171bf897165 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:01.534205 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-0a28f306-0b15-45c8-9cf9-d171bf897165 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:01.535694 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1368/2696] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:37:01 2022] DELETE /identity/v3/projects/aba0546fe49b48b8af71333bfc63aaaa => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:37:01.540034 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a0f606-8e97-4c61-924e-727bd5aa6fd5 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:01.549473 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a0f606-8e97-4c61-924e-727bd5aa6fd5 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:01.567958 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:37:01.605700 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2373cf69-ae1e-44c0-96bf-8fe81dca041a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:01.606252 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2373cf69-ae1e-44c0-96bf-8fe81dca041a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:01.606527 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2373cf69-ae1e-44c0-96bf-8fe81dca041a admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:01.606794 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2373cf69-ae1e-44c0-96bf-8fe81dca041a admin admin] PATH_INFO: `/v3/users/6095d0b6c0494984bafe8799ed083478` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:01.607966 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2373cf69-ae1e-44c0-96bf-8fe81dca041a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6095d0b6c0494984bafe8799ed083478)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:01.610329 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2373cf69-ae1e-44c0-96bf-8fe81dca041a admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:01.765702 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-2373cf69-ae1e-44c0-96bf-8fe81dca041a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6095d0b6c0494984bafe8799ed083478'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:01.777817 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-2373cf69-ae1e-44c0-96bf-8fe81dca041a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6095d0b6c0494984bafe8799ed083478'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:01.821241 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-2373cf69-ae1e-44c0-96bf-8fe81dca041a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6095d0b6c0494984bafe8799ed083478'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:01.821851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-2373cf69-ae1e-44c0-96bf-8fe81dca041a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6095d0b6c0494984bafe8799ed083478'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:01.829021 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-2373cf69-ae1e-44c0-96bf-8fe81dca041a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6095d0b6c0494984bafe8799ed083478'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:01.834919 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1329/2697] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:37:01 2022] DELETE /identity/v3/users/6095d0b6c0494984bafe8799ed083478 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:37:02.204981 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28f306-0b15-45c8-9cf9-d171bf897165 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:02.206454 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-0a28f306-0b15-45c8-9cf9-d171bf897165 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:37:02.222110 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28f306-0b15-45c8-9cf9-d171bf897165 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:02.295873 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ca9e9-4acc-4294-adc5-f2765589c3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:02.296570 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ca9e9-4acc-4294-adc5-f2765589c3a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:02.296850 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ca9e9-4acc-4294-adc5-f2765589c3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:02.297119 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ca9e9-4acc-4294-adc5-f2765589c3a6 admin admin] PATH_INFO: `/v3/projects/1bbe979a71c94f32a84b6ccba3fed5b8` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:02.298297 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ca9e9-4acc-4294-adc5-f2765589c3a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bbe979a71c94f32a84b6ccba3fed5b8)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:02.300204 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ca9e9-4acc-4294-adc5-f2765589c3a6 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:02.392487 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:37:02.409430 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-5f2ca9e9-4acc-4294-adc5-f2765589c3a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bbe979a71c94f32a84b6ccba3fed5b8', 'request_id': 'req-5f2ca9e9-4acc-4294-adc5-f2765589c3a6'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:02.416329 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-5f2ca9e9-4acc-4294-adc5-f2765589c3a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bbe979a71c94f32a84b6ccba3fed5b8', 'request_id': 'req-5f2ca9e9-4acc-4294-adc5-f2765589c3a6'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:02.421680 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-5f2ca9e9-4acc-4294-adc5-f2765589c3a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bbe979a71c94f32a84b6ccba3fed5b8', 'request_id': 'req-5f2ca9e9-4acc-4294-adc5-f2765589c3a6'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:02.493567 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-5f2ca9e9-4acc-4294-adc5-f2765589c3a6 admin admin] The token cache is being invalidate because project 1bbe979a71c94f32a84b6ccba3fed5b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:37:02.493789 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-5f2ca9e9-4acc-4294-adc5-f2765589c3a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:02.494437 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-5f2ca9e9-4acc-4294-adc5-f2765589c3a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:02.495951 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1369/2698] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:37:02 2022] DELETE /identity/v3/projects/1bbe979a71c94f32a84b6ccba3fed5b8 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:37:02.501010 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f80d51-c509-42ca-a1ab-a0cc9ca94ea1 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:02.501433 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f80d51-c509-42ca-a1ab-a0cc9ca94ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:02.501705 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f80d51-c509-42ca-a1ab-a0cc9ca94ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:02.521776 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-a2f80d51-c509-42ca-a1ab-a0cc9ca94ea1 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:37:02.523592 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a2f80d51-c509-42ca-a1ab-a0cc9ca94ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:37:02.586949 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1330/2699] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:37:02.595216 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ca1f0-ba7a-4832-b8b3-621af67fa5f6 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:02.595504 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ca1f0-ba7a-4832-b8b3-621af67fa5f6 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:02.595781 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ca1f0-ba7a-4832-b8b3-621af67fa5f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:02.615664 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-2f5ca1f0-ba7a-4832-b8b3-621af67fa5f6 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:37:02.616933 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-2f5ca1f0-ba7a-4832-b8b3-621af67fa5f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:37:02.624945 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1370/2700] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:37:02.629089 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f80d51-c509-42ca-a1ab-a0cc9ca94ea1 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:02.639179 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f80d51-c509-42ca-a1ab-a0cc9ca94ea1 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:02.643865 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a9b2b6-89aa-48a6-9d7a-d5f10137f23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:02.651502 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a9b2b6-89aa-48a6-9d7a-d5f10137f23f admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:02.651502 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a9b2b6-89aa-48a6-9d7a-d5f10137f23f admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:02.651502 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a9b2b6-89aa-48a6-9d7a-d5f10137f23f admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:02.651502 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a9b2b6-89aa-48a6-9d7a-d5f10137f23f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:37:02.651502 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a9b2b6-89aa-48a6-9d7a-d5f10137f23f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:02.651502 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a9b2b6-89aa-48a6-9d7a-d5f10137f23f admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:02.659925 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1331/2701] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:37:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:37:02.664283 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5ca1f0-ba7a-4832-b8b3-621af67fa5f6 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:02.673484 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5ca1f0-ba7a-4832-b8b3-621af67fa5f6 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:02.678418 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d2b77-86e5-4d0d-a2f4-60f58c3b55c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:02.685913 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d2b77-86e5-4d0d-a2f4-60f58c3b55c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:02.685913 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d2b77-86e5-4d0d-a2f4-60f58c3b55c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:02.685913 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d2b77-86e5-4d0d-a2f4-60f58c3b55c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:02.685913 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8d2b77-86e5-4d0d-a2f4-60f58c3b55c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:02.685913 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8d2b77-86e5-4d0d-a2f4-60f58c3b55c9 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:02.787536 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1371/2702] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:37:02 2022] POST /identity/v3/projects => generated 381 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:37:02.791847 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a9b2b6-89aa-48a6-9d7a-d5f10137f23f admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:02.801878 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a9b2b6-89aa-48a6-9d7a-d5f10137f23f admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:02.827592 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:37:02.858269 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea4bf5-6763-4540-b710-26fb0aa6d8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:02.858871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea4bf5-6763-4540-b710-26fb0aa6d8c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:02.859146 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea4bf5-6763-4540-b710-26fb0aa6d8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:02.859422 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea4bf5-6763-4540-b710-26fb0aa6d8c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:02.860078 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea4bf5-6763-4540-b710-26fb0aa6d8c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:02.862012 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea4bf5-6763-4540-b710-26fb0aa6d8c4 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:02.981055 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1332/2703] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:37:02 2022] POST /identity/v3/users => generated 291 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:37:02.985113 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d2b77-86e5-4d0d-a2f4-60f58c3b55c9 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:02.995501 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d2b77-86e5-4d0d-a2f4-60f58c3b55c9 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:03.000063 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae7a38c-7c21-46dd-b241-dfeeec1cb607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:03.000704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7a38c-7c21-46dd-b241-dfeeec1cb607 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:03.001059 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7a38c-7c21-46dd-b241-dfeeec1cb607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:03.001331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7a38c-7c21-46dd-b241-dfeeec1cb607 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:03.002047 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae7a38c-7c21-46dd-b241-dfeeec1cb607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:03.003605 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae7a38c-7c21-46dd-b241-dfeeec1cb607 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:03.014072 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1372/2704] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:37:03.025737 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea4bf5-6763-4540-b710-26fb0aa6d8c4 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:03.035776 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea4bf5-6763-4540-b710-26fb0aa6d8c4 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:03.040876 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded9e13-b297-4711-88ee-ef2ef5f090af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:03.041416 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded9e13-b297-4711-88ee-ef2ef5f090af admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:03.041690 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded9e13-b297-4711-88ee-ef2ef5f090af admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:03.042006 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded9e13-b297-4711-88ee-ef2ef5f090af admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:03.042687 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eded9e13-b297-4711-88ee-ef2ef5f090af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:03.044254 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eded9e13-b297-4711-88ee-ef2ef5f090af admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:03.054059 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1333/2705] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:37:03.058552 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae7a38c-7c21-46dd-b241-dfeeec1cb607 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:03.068603 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae7a38c-7c21-46dd-b241-dfeeec1cb607 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:03.074130 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41bb874-c4c7-44b3-8900-0ffc19b911cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:03.074785 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41bb874-c4c7-44b3-8900-0ffc19b911cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:03.075062 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41bb874-c4c7-44b3-8900-0ffc19b911cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:03.075332 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41bb874-c4c7-44b3-8900-0ffc19b911cd admin admin] PATH_INFO: `/v3/projects/f411003721854ceba7d2cceda92b1e15/users/c03efd2e0b5c4b48951f83d1383791b1/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:03.089930 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41bb874-c4c7-44b3-8900-0ffc19b911cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f411003721854ceba7d2cceda92b1e15, user_id=c03efd2e0b5c4b48951f83d1383791b1, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:03.092699 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41bb874-c4c7-44b3-8900-0ffc19b911cd admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:03.163089 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1373/2706] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:37:03 2022] PUT /identity/v3/projects/f411003721854ceba7d2cceda92b1e15/users/c03efd2e0b5c4b48951f83d1383791b1/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:37:03.174869 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f998c95-80c4-41c0-ad7f-b63ee9a99de7 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:03.175173 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f998c95-80c4-41c0-ad7f-b63ee9a99de7 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:03.175445 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f998c95-80c4-41c0-ad7f-b63ee9a99de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:03.205625 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-0f998c95-80c4-41c0-ad7f-b63ee9a99de7 None None] MFA Rules not processed for user `c03efd2e0b5c4b48951f83d1383791b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:37:03.206851 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-0f998c95-80c4-41c0-ad7f-b63ee9a99de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:37:03.269912 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1334/2707] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:37:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:37:03.280922 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41bb874-c4c7-44b3-8900-0ffc19b911cd admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:03.282776 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-f41bb874-c4c7-44b3-8900-0ffc19b911cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:37:03.344222 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41bb874-c4c7-44b3-8900-0ffc19b911cd admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:03.347827 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2966cf37-0b6d-4fc2-8d5c-86a63249ca45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:03.348467 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966cf37-0b6d-4fc2-8d5c-86a63249ca45 service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:03.348760 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966cf37-0b6d-4fc2-8d5c-86a63249ca45 service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:03.349030 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966cf37-0b6d-4fc2-8d5c-86a63249ca45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:03.357255 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2966cf37-0b6d-4fc2-8d5c-86a63249ca45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:03.358679 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2966cf37-0b6d-4fc2-8d5c-86a63249ca45 service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:03.368092 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1374/2708] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:37:03 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:37:03.388638 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f998c95-80c4-41c0-ad7f-b63ee9a99de7 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:03.460435 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f998c95-80c4-41c0-ad7f-b63ee9a99de7 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:03.465174 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da880b-3289-4618-bae1-c61945bdc869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:03.465818 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da880b-3289-4618-bae1-c61945bdc869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:03.466102 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da880b-3289-4618-bae1-c61945bdc869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:03.466366 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da880b-3289-4618-bae1-c61945bdc869 admin admin] PATH_INFO: `/v3/users/c03efd2e0b5c4b48951f83d1383791b1` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:03.467604 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22da880b-3289-4618-bae1-c61945bdc869 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c03efd2e0b5c4b48951f83d1383791b1)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:03.469825 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22da880b-3289-4618-bae1-c61945bdc869 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:03.669292 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:37:03.675033 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-22da880b-3289-4618-bae1-c61945bdc869 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c03efd2e0b5c4b48951f83d1383791b1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:03.687435 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-22da880b-3289-4618-bae1-c61945bdc869 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c03efd2e0b5c4b48951f83d1383791b1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:03.774067 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-22da880b-3289-4618-bae1-c61945bdc869 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c03efd2e0b5c4b48951f83d1383791b1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:03.774690 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-22da880b-3289-4618-bae1-c61945bdc869 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c03efd2e0b5c4b48951f83d1383791b1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:03.782023 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-22da880b-3289-4618-bae1-c61945bdc869 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c03efd2e0b5c4b48951f83d1383791b1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:03.788046 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1335/2709] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:37:03 2022] DELETE /identity/v3/users/c03efd2e0b5c4b48951f83d1383791b1 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:37:03.793461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bedfa80-0fbe-4f91-827f-a3e551f410e6 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:03.795265 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bedfa80-0fbe-4f91-827f-a3e551f410e6 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:03.795265 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bedfa80-0fbe-4f91-827f-a3e551f410e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:03.812881 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-7bedfa80-0fbe-4f91-827f-a3e551f410e6 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:37:03.814123 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-7bedfa80-0fbe-4f91-827f-a3e551f410e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:37:03.864728 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:37:03.876733 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1375/2710] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:37:03.886784 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da880b-3289-4618-bae1-c61945bdc869 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:03.888217 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-22da880b-3289-4618-bae1-c61945bdc869 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:37:03.946073 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da880b-3289-4618-bae1-c61945bdc869 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:03.949627 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b828367-d121-4172-be40-b47a8427b0ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:03.952257 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b828367-d121-4172-be40-b47a8427b0ef service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:03.952257 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b828367-d121-4172-be40-b47a8427b0ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:03.952257 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b828367-d121-4172-be40-b47a8427b0ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:03.959586 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b828367-d121-4172-be40-b47a8427b0ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:03.961819 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b828367-d121-4172-be40-b47a8427b0ef service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:03.971322 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1336/2711] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:37:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:37:04.371044 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bedfa80-0fbe-4f91-827f-a3e551f410e6 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:04.372593 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-7bedfa80-0fbe-4f91-827f-a3e551f410e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:37:04.385307 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bedfa80-0fbe-4f91-827f-a3e551f410e6 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:04.390025 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48316b39-d9ff-4ce9-8881-ef8235cff198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:04.390683 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48316b39-d9ff-4ce9-8881-ef8235cff198 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:04.390957 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48316b39-d9ff-4ce9-8881-ef8235cff198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:04.391233 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48316b39-d9ff-4ce9-8881-ef8235cff198 admin admin] PATH_INFO: `/v3/projects/f411003721854ceba7d2cceda92b1e15` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:04.392437 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48316b39-d9ff-4ce9-8881-ef8235cff198 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f411003721854ceba7d2cceda92b1e15)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:04.394360 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48316b39-d9ff-4ce9-8881-ef8235cff198 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:04.506592 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-48316b39-d9ff-4ce9-8881-ef8235cff198 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f411003721854ceba7d2cceda92b1e15', 'request_id': 'req-48316b39-d9ff-4ce9-8881-ef8235cff198'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:04.513255 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-48316b39-d9ff-4ce9-8881-ef8235cff198 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f411003721854ceba7d2cceda92b1e15', 'request_id': 'req-48316b39-d9ff-4ce9-8881-ef8235cff198'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:04.519402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-48316b39-d9ff-4ce9-8881-ef8235cff198 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f411003721854ceba7d2cceda92b1e15', 'request_id': 'req-48316b39-d9ff-4ce9-8881-ef8235cff198'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:04.614565 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-48316b39-d9ff-4ce9-8881-ef8235cff198 admin admin] The token cache is being invalidate because project f411003721854ceba7d2cceda92b1e15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:37:04.614856 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-48316b39-d9ff-4ce9-8881-ef8235cff198 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:04.615483 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-48316b39-d9ff-4ce9-8881-ef8235cff198 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:04.617819 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1376/2712] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:37:04 2022] DELETE /identity/v3/projects/f411003721854ceba7d2cceda92b1e15 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:37:04.622722 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b28e3d-fa24-4200-af65-6e08abc37c21 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:04.623010 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b28e3d-fa24-4200-af65-6e08abc37c21 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:04.623363 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b28e3d-fa24-4200-af65-6e08abc37c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:04.644345 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-c1b28e3d-fa24-4200-af65-6e08abc37c21 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:37:04.645604 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c1b28e3d-fa24-4200-af65-6e08abc37c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:37:04.706489 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1337/2713] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:37:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:37:04.714445 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25958558-495c-41de-80ae-b6f21a22a039 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:04.714737 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25958558-495c-41de-80ae-b6f21a22a039 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:04.715019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25958558-495c-41de-80ae-b6f21a22a039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:04.734990 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-25958558-495c-41de-80ae-b6f21a22a039 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:37:04.736254 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-25958558-495c-41de-80ae-b6f21a22a039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:37:04.744876 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1377/2714] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:37:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:37:04.749169 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b28e3d-fa24-4200-af65-6e08abc37c21 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:04.764224 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b28e3d-fa24-4200-af65-6e08abc37c21 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:04.769356 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2834f6-838f-4710-a8d4-0f8fb49d9192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:04.770048 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2834f6-838f-4710-a8d4-0f8fb49d9192 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:04.770331 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2834f6-838f-4710-a8d4-0f8fb49d9192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:04.770618 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2834f6-838f-4710-a8d4-0f8fb49d9192 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:04.771329 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2834f6-838f-4710-a8d4-0f8fb49d9192 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:37:04.771631 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2834f6-838f-4710-a8d4-0f8fb49d9192 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:04.773301 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2834f6-838f-4710-a8d4-0f8fb49d9192 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:04.785379 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1338/2715] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:37:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:37:04.789768 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25958558-495c-41de-80ae-b6f21a22a039 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:04.798983 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25958558-495c-41de-80ae-b6f21a22a039 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:04.803831 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d0e7da-51e7-49c1-9e71-150a2d9619fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:04.805148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0e7da-51e7-49c1-9e71-150a2d9619fb admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:04.805578 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0e7da-51e7-49c1-9e71-150a2d9619fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:04.805861 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0e7da-51e7-49c1-9e71-150a2d9619fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:04.806527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d0e7da-51e7-49c1-9e71-150a2d9619fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:04.808424 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d0e7da-51e7-49c1-9e71-150a2d9619fb admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:04.907544 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1378/2716] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:37:04 2022] POST /identity/v3/projects => generated 375 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:37:04.911622 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2834f6-838f-4710-a8d4-0f8fb49d9192 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:04.922389 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2834f6-838f-4710-a8d4-0f8fb49d9192 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:04.965694 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:37:04.978935 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7faf84-7d6e-48d0-b068-51ec50e477fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:04.979553 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7faf84-7d6e-48d0-b068-51ec50e477fd admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:04.981885 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7faf84-7d6e-48d0-b068-51ec50e477fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:04.981885 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7faf84-7d6e-48d0-b068-51ec50e477fd admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:04.981885 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7faf84-7d6e-48d0-b068-51ec50e477fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:04.982986 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7faf84-7d6e-48d0-b068-51ec50e477fd admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:05.072183 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1339/2717] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:37:04 2022] POST /identity/v3/users => generated 288 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:37:05.076296 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d0e7da-51e7-49c1-9e71-150a2d9619fb admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:05.086739 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d0e7da-51e7-49c1-9e71-150a2d9619fb admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:05.091260 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9bc55a-998a-4bd8-934d-6ae34c006104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:05.100987 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9bc55a-998a-4bd8-934d-6ae34c006104 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:05.100987 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9bc55a-998a-4bd8-934d-6ae34c006104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:05.100987 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9bc55a-998a-4bd8-934d-6ae34c006104 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:05.100987 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9bc55a-998a-4bd8-934d-6ae34c006104 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:05.100987 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9bc55a-998a-4bd8-934d-6ae34c006104 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:05.105548 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1379/2718] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:37:05.109384 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7faf84-7d6e-48d0-b068-51ec50e477fd admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:05.120295 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7faf84-7d6e-48d0-b068-51ec50e477fd admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:05.125095 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f38662-d88c-441d-ba29-9f625660028e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:05.125702 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f38662-d88c-441d-ba29-9f625660028e admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:05.125975 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f38662-d88c-441d-ba29-9f625660028e admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:05.126261 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f38662-d88c-441d-ba29-9f625660028e admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:05.126967 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f38662-d88c-441d-ba29-9f625660028e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:05.128576 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f38662-d88c-441d-ba29-9f625660028e admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:05.139123 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1340/2719] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:37:05.143355 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9bc55a-998a-4bd8-934d-6ae34c006104 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:05.153330 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9bc55a-998a-4bd8-934d-6ae34c006104 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:05.157900 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a31d7f-9c08-4413-8f86-957b9c4a4a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:05.158555 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a31d7f-9c08-4413-8f86-957b9c4a4a68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:05.158917 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a31d7f-9c08-4413-8f86-957b9c4a4a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:05.159204 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a31d7f-9c08-4413-8f86-957b9c4a4a68 admin admin] PATH_INFO: `/v3/projects/bfa9d6def8254bcdae820dd465091e7d/users/e354173992444793aa6fe7eb58542c1c/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:05.174628 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a31d7f-9c08-4413-8f86-957b9c4a4a68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfa9d6def8254bcdae820dd465091e7d, user_id=e354173992444793aa6fe7eb58542c1c, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:05.177321 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a31d7f-9c08-4413-8f86-957b9c4a4a68 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:05.254500 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1380/2720] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:37:05 2022] PUT /identity/v3/projects/bfa9d6def8254bcdae820dd465091e7d/users/e354173992444793aa6fe7eb58542c1c/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:37:05.265222 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da5c87c-b871-45f7-9b86-962858b5ba1d None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:05.265504 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da5c87c-b871-45f7-9b86-962858b5ba1d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:05.265780 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da5c87c-b871-45f7-9b86-962858b5ba1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:05.294445 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-8da5c87c-b871-45f7-9b86-962858b5ba1d None None] MFA Rules not processed for user `e354173992444793aa6fe7eb58542c1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:37:05.295634 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8da5c87c-b871-45f7-9b86-962858b5ba1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:37:05.358449 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1341/2721] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:37:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:37:05.369670 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a31d7f-9c08-4413-8f86-957b9c4a4a68 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:05.371385 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-46a31d7f-9c08-4413-8f86-957b9c4a4a68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:37:05.427775 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a31d7f-9c08-4413-8f86-957b9c4a4a68 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:05.431437 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efaf77c-1d89-48e6-b317-a7f698b93ba6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:05.432049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efaf77c-1d89-48e6-b317-a7f698b93ba6 service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:05.432331 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efaf77c-1d89-48e6-b317-a7f698b93ba6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:05.432633 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efaf77c-1d89-48e6-b317-a7f698b93ba6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:05.440924 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efaf77c-1d89-48e6-b317-a7f698b93ba6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:05.442364 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efaf77c-1d89-48e6-b317-a7f698b93ba6 service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:05.451254 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1381/2722] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:37:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:37:15.355566 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=45,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:37:15.447265 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:37:31.272171 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da5c87c-b871-45f7-9b86-962858b5ba1d None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:31.333520 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da5c87c-b871-45f7-9b86-962858b5ba1d None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:31.338801 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324cb742-bc56-4c68-9ae2-8efddbf52613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:31.339447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324cb742-bc56-4c68-9ae2-8efddbf52613 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:31.339726 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324cb742-bc56-4c68-9ae2-8efddbf52613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:31.340029 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324cb742-bc56-4c68-9ae2-8efddbf52613 admin admin] PATH_INFO: `/v3/users/e354173992444793aa6fe7eb58542c1c` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:31.341213 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324cb742-bc56-4c68-9ae2-8efddbf52613 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e354173992444793aa6fe7eb58542c1c)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:31.343218 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324cb742-bc56-4c68-9ae2-8efddbf52613 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:31.490501 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-324cb742-bc56-4c68-9ae2-8efddbf52613 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e354173992444793aa6fe7eb58542c1c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:31.502810 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-324cb742-bc56-4c68-9ae2-8efddbf52613 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e354173992444793aa6fe7eb58542c1c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:31.558200 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-324cb742-bc56-4c68-9ae2-8efddbf52613 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e354173992444793aa6fe7eb58542c1c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:31.558818 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-324cb742-bc56-4c68-9ae2-8efddbf52613 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e354173992444793aa6fe7eb58542c1c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:31.566398 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-324cb742-bc56-4c68-9ae2-8efddbf52613 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e354173992444793aa6fe7eb58542c1c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:31.572339 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1342/2723] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:37:31 2022] DELETE /identity/v3/users/e354173992444793aa6fe7eb58542c1c => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:37:31.577063 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e51c8-14f3-4932-994d-d5c2e4cd66dd None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:31.577549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e51c8-14f3-4932-994d-d5c2e4cd66dd None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:31.577658 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e51c8-14f3-4932-994d-d5c2e4cd66dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:31.596374 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-383e51c8-14f3-4932-994d-d5c2e4cd66dd None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:37:31.597596 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-383e51c8-14f3-4932-994d-d5c2e4cd66dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:37:31.656478 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1382/2724] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:37:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:37:31.666471 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324cb742-bc56-4c68-9ae2-8efddbf52613 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:31.667878 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-324cb742-bc56-4c68-9ae2-8efddbf52613 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:37:31.727683 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324cb742-bc56-4c68-9ae2-8efddbf52613 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:31.731418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6ee6a-74d2-458f-ac2c-2cabdfddf9d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:31.732028 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6ee6a-74d2-458f-ac2c-2cabdfddf9d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:31.732304 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6ee6a-74d2-458f-ac2c-2cabdfddf9d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:31.732606 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6ee6a-74d2-458f-ac2c-2cabdfddf9d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:31.741594 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba6ee6a-74d2-458f-ac2c-2cabdfddf9d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:31.742950 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba6ee6a-74d2-458f-ac2c-2cabdfddf9d7 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:31.752244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1343/2725] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:37:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:37:32.158946 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e51c8-14f3-4932-994d-d5c2e4cd66dd None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:32.160685 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-383e51c8-14f3-4932-994d-d5c2e4cd66dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:37:32.172922 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e51c8-14f3-4932-994d-d5c2e4cd66dd None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:32.177612 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb4b16-c8e9-4089-8a2f-e19c059ed845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:32.178256 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb4b16-c8e9-4089-8a2f-e19c059ed845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:32.178544 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb4b16-c8e9-4089-8a2f-e19c059ed845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:32.178804 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb4b16-c8e9-4089-8a2f-e19c059ed845 admin admin] PATH_INFO: `/v3/projects/bfa9d6def8254bcdae820dd465091e7d` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:32.180082 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cb4b16-c8e9-4089-8a2f-e19c059ed845 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfa9d6def8254bcdae820dd465091e7d)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:32.183126 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cb4b16-c8e9-4089-8a2f-e19c059ed845 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:32.281778 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-95cb4b16-c8e9-4089-8a2f-e19c059ed845 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfa9d6def8254bcdae820dd465091e7d', 'request_id': 'req-95cb4b16-c8e9-4089-8a2f-e19c059ed845'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:32.287898 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-95cb4b16-c8e9-4089-8a2f-e19c059ed845 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfa9d6def8254bcdae820dd465091e7d', 'request_id': 'req-95cb4b16-c8e9-4089-8a2f-e19c059ed845'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:32.294370 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-95cb4b16-c8e9-4089-8a2f-e19c059ed845 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfa9d6def8254bcdae820dd465091e7d', 'request_id': 'req-95cb4b16-c8e9-4089-8a2f-e19c059ed845'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:32.379566 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-95cb4b16-c8e9-4089-8a2f-e19c059ed845 admin admin] The token cache is being invalidate because project bfa9d6def8254bcdae820dd465091e7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:37:32.379851 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-95cb4b16-c8e9-4089-8a2f-e19c059ed845 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:32.380491 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-95cb4b16-c8e9-4089-8a2f-e19c059ed845 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:32.381992 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1383/2726] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:37:32 2022] DELETE /identity/v3/projects/bfa9d6def8254bcdae820dd465091e7d => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:37:32.387033 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386953e4-6162-4051-a332-f6b75637ec98 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:32.387510 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386953e4-6162-4051-a332-f6b75637ec98 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:32.387635 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386953e4-6162-4051-a332-f6b75637ec98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:32.408271 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-386953e4-6162-4051-a332-f6b75637ec98 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:37:32.409501 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-386953e4-6162-4051-a332-f6b75637ec98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:37:32.426456 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:37:32.468107 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1344/2727] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:37:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:37:32.476178 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed5d79-2642-48ef-bbea-7bfb3811647d None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:32.476491 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed5d79-2642-48ef-bbea-7bfb3811647d None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:32.476770 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed5d79-2642-48ef-bbea-7bfb3811647d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:32.497982 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-beed5d79-2642-48ef-bbea-7bfb3811647d None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:37:32.499175 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-beed5d79-2642-48ef-bbea-7bfb3811647d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:37:32.506947 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1384/2728] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:37:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:37:32.511801 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386953e4-6162-4051-a332-f6b75637ec98 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:32.521590 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386953e4-6162-4051-a332-f6b75637ec98 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:32.526189 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823219e9-5216-4715-8e2b-3e11ccd856d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:32.526793 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823219e9-5216-4715-8e2b-3e11ccd856d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:32.527065 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823219e9-5216-4715-8e2b-3e11ccd856d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:32.527333 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823219e9-5216-4715-8e2b-3e11ccd856d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:32.528018 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823219e9-5216-4715-8e2b-3e11ccd856d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:37:32.528316 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823219e9-5216-4715-8e2b-3e11ccd856d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:32.529924 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823219e9-5216-4715-8e2b-3e11ccd856d0 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:32.541878 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1345/2729] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:37:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:37:32.546138 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed5d79-2642-48ef-bbea-7bfb3811647d None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:32.555478 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed5d79-2642-48ef-bbea-7bfb3811647d None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:32.560984 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e345a-6c26-4061-8413-814635c46900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:32.561592 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e345a-6c26-4061-8413-814635c46900 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:32.561868 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e345a-6c26-4061-8413-814635c46900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:32.562143 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e345a-6c26-4061-8413-814635c46900 admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:32.562805 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99e345a-6c26-4061-8413-814635c46900 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:32.564715 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99e345a-6c26-4061-8413-814635c46900 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:32.643704 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1385/2730] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:37:32 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:37:32.647977 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823219e9-5216-4715-8e2b-3e11ccd856d0 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:32.658651 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823219e9-5216-4715-8e2b-3e11ccd856d0 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:32.714473 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0581e01-0961-466d-ba89-f865afd21ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:32.715078 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0581e01-0961-466d-ba89-f865afd21ab7 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:32.715354 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0581e01-0961-466d-ba89-f865afd21ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:32.715625 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0581e01-0961-466d-ba89-f865afd21ab7 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:32.716280 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0581e01-0961-466d-ba89-f865afd21ab7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:32.718205 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0581e01-0961-466d-ba89-f865afd21ab7 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:32.802878 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1346/2731] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:37:32 2022] POST /identity/v3/users => generated 287 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:37:32.807556 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e345a-6c26-4061-8413-814635c46900 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:32.817871 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e345a-6c26-4061-8413-814635c46900 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:32.822403 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adec328-1bed-407e-ae52-973c6e71baea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:32.823654 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adec328-1bed-407e-ae52-973c6e71baea admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:32.823925 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adec328-1bed-407e-ae52-973c6e71baea admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:32.824190 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adec328-1bed-407e-ae52-973c6e71baea admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:32.824910 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adec328-1bed-407e-ae52-973c6e71baea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:32.826483 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adec328-1bed-407e-ae52-973c6e71baea admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:32.836095 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1386/2732] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:37:32.840676 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0581e01-0961-466d-ba89-f865afd21ab7 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:32.850786 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0581e01-0961-466d-ba89-f865afd21ab7 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:32.855403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a604068-4f19-4496-90f5-327529718f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:32.856003 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a604068-4f19-4496-90f5-327529718f75 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:32.856280 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a604068-4f19-4496-90f5-327529718f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:32.856590 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a604068-4f19-4496-90f5-327529718f75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:32.857274 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a604068-4f19-4496-90f5-327529718f75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:32.858846 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a604068-4f19-4496-90f5-327529718f75 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:32.869235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1347/2733] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:37:32.874228 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adec328-1bed-407e-ae52-973c6e71baea admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:32.884234 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adec328-1bed-407e-ae52-973c6e71baea admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:32.889628 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ffdf9-3729-48c6-82c1-5125c6cc9a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:32.890289 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ffdf9-3729-48c6-82c1-5125c6cc9a42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:32.890566 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ffdf9-3729-48c6-82c1-5125c6cc9a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:32.890836 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ffdf9-3729-48c6-82c1-5125c6cc9a42 admin admin] PATH_INFO: `/v3/projects/2174818a868a4fd39aacb3090c8de2aa/users/a0ad8ccb8bad4a53906bb05b7ddfb923/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:32.905135 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ffdf9-3729-48c6-82c1-5125c6cc9a42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2174818a868a4fd39aacb3090c8de2aa, user_id=a0ad8ccb8bad4a53906bb05b7ddfb923, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:32.907771 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ffdf9-3729-48c6-82c1-5125c6cc9a42 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:32.957010 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1387/2734] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:37:32 2022] PUT /identity/v3/projects/2174818a868a4fd39aacb3090c8de2aa/users/a0ad8ccb8bad4a53906bb05b7ddfb923/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:37:32.967426 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5531478-9ee1-4941-b08d-af0700ed50f1 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:32.967701 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5531478-9ee1-4941-b08d-af0700ed50f1 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:32.967967 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5531478-9ee1-4941-b08d-af0700ed50f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:32.996900 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-b5531478-9ee1-4941-b08d-af0700ed50f1 None None] MFA Rules not processed for user `a0ad8ccb8bad4a53906bb05b7ddfb923`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:37:32.998209 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-b5531478-9ee1-4941-b08d-af0700ed50f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:37:33.061669 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1348/2735] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:37:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:37:33.072250 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ffdf9-3729-48c6-82c1-5125c6cc9a42 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:33.074072 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-797ffdf9-3729-48c6-82c1-5125c6cc9a42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:37:33.096073 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:37:33.131603 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ffdf9-3729-48c6-82c1-5125c6cc9a42 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:33.135572 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeaff82-1b31-4690-9e64-e7db2bbc52f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:33.136292 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeaff82-1b31-4690-9e64-e7db2bbc52f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:33.136599 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeaff82-1b31-4690-9e64-e7db2bbc52f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:33.136870 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeaff82-1b31-4690-9e64-e7db2bbc52f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:33.145019 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeaff82-1b31-4690-9e64-e7db2bbc52f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:33.146433 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeaff82-1b31-4690-9e64-e7db2bbc52f2 service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:33.155123 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1388/2736] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:37:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:37:37.031684 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5531478-9ee1-4941-b08d-af0700ed50f1 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:37.051185 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:37:37.096696 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5531478-9ee1-4941-b08d-af0700ed50f1 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:37.101186 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6009de-4bf5-4d63-9f24-9df69208490f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:37.101812 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6009de-4bf5-4d63-9f24-9df69208490f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:37.102084 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6009de-4bf5-4d63-9f24-9df69208490f admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:37.102348 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6009de-4bf5-4d63-9f24-9df69208490f admin admin] PATH_INFO: `/v3/users/a0ad8ccb8bad4a53906bb05b7ddfb923` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:37.103471 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6009de-4bf5-4d63-9f24-9df69208490f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0ad8ccb8bad4a53906bb05b7ddfb923)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:37.105997 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6009de-4bf5-4d63-9f24-9df69208490f admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:37.261537 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ce6009de-4bf5-4d63-9f24-9df69208490f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0ad8ccb8bad4a53906bb05b7ddfb923'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:37.273155 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ce6009de-4bf5-4d63-9f24-9df69208490f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0ad8ccb8bad4a53906bb05b7ddfb923'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:37.350694 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ce6009de-4bf5-4d63-9f24-9df69208490f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0ad8ccb8bad4a53906bb05b7ddfb923'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:37.351305 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ce6009de-4bf5-4d63-9f24-9df69208490f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0ad8ccb8bad4a53906bb05b7ddfb923'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:37.359468 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ce6009de-4bf5-4d63-9f24-9df69208490f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0ad8ccb8bad4a53906bb05b7ddfb923'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:37.365477 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1349/2737] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:37:37 2022] DELETE /identity/v3/users/a0ad8ccb8bad4a53906bb05b7ddfb923 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:37:37.370064 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f7fad-fd55-4dee-af69-f837fd054fb0 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:37.370582 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f7fad-fd55-4dee-af69-f837fd054fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:37.370687 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f7fad-fd55-4dee-af69-f837fd054fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:37.389287 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-342f7fad-fd55-4dee-af69-f837fd054fb0 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:37:37.390475 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-342f7fad-fd55-4dee-af69-f837fd054fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:37:37.448128 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1389/2738] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:37:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:37:37.458666 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6009de-4bf5-4d63-9f24-9df69208490f admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:37.460039 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ce6009de-4bf5-4d63-9f24-9df69208490f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:37:37.517054 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6009de-4bf5-4d63-9f24-9df69208490f admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:37.520873 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2345f9f-3cd6-48ce-88e0-656548071914 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:37.521485 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2345f9f-3cd6-48ce-88e0-656548071914 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:37.521758 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2345f9f-3cd6-48ce-88e0-656548071914 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:37.522024 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2345f9f-3cd6-48ce-88e0-656548071914 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:37.532557 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2345f9f-3cd6-48ce-88e0-656548071914 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:37.533967 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2345f9f-3cd6-48ce-88e0-656548071914 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:37.543743 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1350/2739] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:37:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:37:37.958009 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342f7fad-fd55-4dee-af69-f837fd054fb0 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:37.959509 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-342f7fad-fd55-4dee-af69-f837fd054fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:37:37.972809 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342f7fad-fd55-4dee-af69-f837fd054fb0 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:37.977627 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b108f-9e18-469a-b025-71cce70ee7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:37.978529 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b108f-9e18-469a-b025-71cce70ee7fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:37.978809 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b108f-9e18-469a-b025-71cce70ee7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:37.979083 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b108f-9e18-469a-b025-71cce70ee7fd admin admin] PATH_INFO: `/v3/projects/2174818a868a4fd39aacb3090c8de2aa` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:37.980261 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b108f-9e18-469a-b025-71cce70ee7fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=2174818a868a4fd39aacb3090c8de2aa)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:37.982252 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b108f-9e18-469a-b025-71cce70ee7fd admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:38.095048 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-547b108f-9e18-469a-b025-71cce70ee7fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2174818a868a4fd39aacb3090c8de2aa', 'request_id': 'req-547b108f-9e18-469a-b025-71cce70ee7fd'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:38.101033 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-547b108f-9e18-469a-b025-71cce70ee7fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2174818a868a4fd39aacb3090c8de2aa', 'request_id': 'req-547b108f-9e18-469a-b025-71cce70ee7fd'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:38.106586 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-547b108f-9e18-469a-b025-71cce70ee7fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2174818a868a4fd39aacb3090c8de2aa', 'request_id': 'req-547b108f-9e18-469a-b025-71cce70ee7fd'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:38.162036 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-547b108f-9e18-469a-b025-71cce70ee7fd admin admin] The token cache is being invalidate because project 2174818a868a4fd39aacb3090c8de2aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:37:38.162321 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-547b108f-9e18-469a-b025-71cce70ee7fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:38.162963 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-547b108f-9e18-469a-b025-71cce70ee7fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:37:38.164526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1390/2740] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:37:37 2022] DELETE /identity/v3/projects/2174818a868a4fd39aacb3090c8de2aa => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:37:38.169373 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818094c6-7ee8-4fd1-8f4b-700a7bd92526 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:38.169865 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818094c6-7ee8-4fd1-8f4b-700a7bd92526 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:38.169992 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818094c6-7ee8-4fd1-8f4b-700a7bd92526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:38.189512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-818094c6-7ee8-4fd1-8f4b-700a7bd92526 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:37:38.190708 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-818094c6-7ee8-4fd1-8f4b-700a7bd92526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:37:38.223970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:37:38.254264 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1351/2741] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:37:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:37:38.260139 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526457e5-82e2-4b30-97af-e3f184ac7cf2 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:38.260454 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526457e5-82e2-4b30-97af-e3f184ac7cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:38.260740 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526457e5-82e2-4b30-97af-e3f184ac7cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:38.281027 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-526457e5-82e2-4b30-97af-e3f184ac7cf2 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:37:38.282281 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-526457e5-82e2-4b30-97af-e3f184ac7cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:37:38.290557 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1391/2742] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:37:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:37:38.295483 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818094c6-7ee8-4fd1-8f4b-700a7bd92526 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:38.311546 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818094c6-7ee8-4fd1-8f4b-700a7bd92526 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:38.316214 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d2cda-0257-46f8-b28b-65f819bd21ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:38.317072 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d2cda-0257-46f8-b28b-65f819bd21ca admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:38.317514 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d2cda-0257-46f8-b28b-65f819bd21ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:38.317768 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d2cda-0257-46f8-b28b-65f819bd21ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:38.318768 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93d2cda-0257-46f8-b28b-65f819bd21ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:37:38.319071 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93d2cda-0257-46f8-b28b-65f819bd21ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:38.320693 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93d2cda-0257-46f8-b28b-65f819bd21ca admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:38.332876 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1352/2743] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:37:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:37:38.337442 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526457e5-82e2-4b30-97af-e3f184ac7cf2 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:38.346968 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526457e5-82e2-4b30-97af-e3f184ac7cf2 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:38.351592 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f979c2-d505-4a21-85a9-f766f1fcdb0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:38.352195 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f979c2-d505-4a21-85a9-f766f1fcdb0c admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:38.352504 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f979c2-d505-4a21-85a9-f766f1fcdb0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:38.352779 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f979c2-d505-4a21-85a9-f766f1fcdb0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:38.353445 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f979c2-d505-4a21-85a9-f766f1fcdb0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:38.356141 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f979c2-d505-4a21-85a9-f766f1fcdb0c admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:38.471976 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1392/2744] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:37:38 2022] POST /identity/v3/projects => generated 371 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:37:38.476373 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d2cda-0257-46f8-b28b-65f819bd21ca admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:38.487234 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d2cda-0257-46f8-b28b-65f819bd21ca admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:38.546330 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce81995-61b8-48f0-8c22-454b58bf439f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:38.546934 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce81995-61b8-48f0-8c22-454b58bf439f admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:38.547211 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce81995-61b8-48f0-8c22-454b58bf439f admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:38.547477 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce81995-61b8-48f0-8c22-454b58bf439f admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:38.548130 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce81995-61b8-48f0-8c22-454b58bf439f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:38.550047 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce81995-61b8-48f0-8c22-454b58bf439f admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:38.637211 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1353/2745] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:37:38 2022] POST /identity/v3/users => generated 286 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:37:38.641515 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f979c2-d505-4a21-85a9-f766f1fcdb0c admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:38.652275 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f979c2-d505-4a21-85a9-f766f1fcdb0c admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:38.656977 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da34e674-dedd-4d63-8703-5ebf0abd6d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:38.657574 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34e674-dedd-4d63-8703-5ebf0abd6d24 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:38.657870 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34e674-dedd-4d63-8703-5ebf0abd6d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:38.658137 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34e674-dedd-4d63-8703-5ebf0abd6d24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:38.658817 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da34e674-dedd-4d63-8703-5ebf0abd6d24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:38.660387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da34e674-dedd-4d63-8703-5ebf0abd6d24 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:38.670883 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1393/2746] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:37:38.674977 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce81995-61b8-48f0-8c22-454b58bf439f admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:38.685328 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce81995-61b8-48f0-8c22-454b58bf439f admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:38.689942 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea49d09-3340-4047-bb86-ca8d3d00ff00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:38.690546 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea49d09-3340-4047-bb86-ca8d3d00ff00 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:38.690826 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea49d09-3340-4047-bb86-ca8d3d00ff00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:38.691100 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea49d09-3340-4047-bb86-ca8d3d00ff00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:38.691789 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea49d09-3340-4047-bb86-ca8d3d00ff00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:38.693428 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea49d09-3340-4047-bb86-ca8d3d00ff00 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:38.703095 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1354/2747] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:37:38.707077 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da34e674-dedd-4d63-8703-5ebf0abd6d24 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:38.717411 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da34e674-dedd-4d63-8703-5ebf0abd6d24 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:38.722188 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2b1b8-e50f-42d5-84e4-87a5daa24868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:38.722829 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2b1b8-e50f-42d5-84e4-87a5daa24868 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:38.723108 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2b1b8-e50f-42d5-84e4-87a5daa24868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:38.723376 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2b1b8-e50f-42d5-84e4-87a5daa24868 admin admin] PATH_INFO: `/v3/projects/58c88a9de1de4e2b83bb8a37142608a6/users/4552c0df2a3f4f30b6b2829de3a8284d/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:38.725552 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:37:38.738648 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab2b1b8-e50f-42d5-84e4-87a5daa24868 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58c88a9de1de4e2b83bb8a37142608a6, user_id=4552c0df2a3f4f30b6b2829de3a8284d, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:38.741319 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab2b1b8-e50f-42d5-84e4-87a5daa24868 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:38.853322 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1394/2748] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:37:38 2022] PUT /identity/v3/projects/58c88a9de1de4e2b83bb8a37142608a6/users/4552c0df2a3f4f30b6b2829de3a8284d/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:37:38.859160 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6527fe-dcf5-4e38-ba8d-e26668af29d5 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:38.859478 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6527fe-dcf5-4e38-ba8d-e26668af29d5 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:38.869122 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6527fe-dcf5-4e38-ba8d-e26668af29d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:38.878643 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-7c6527fe-dcf5-4e38-ba8d-e26668af29d5 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:37:38.879837 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-7c6527fe-dcf5-4e38-ba8d-e26668af29d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:37:38.937677 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1355/2749] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:37:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:37:38.947449 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2b1b8-e50f-42d5-84e4-87a5daa24868 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:38.949818 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-4ab2b1b8-e50f-42d5-84e4-87a5daa24868 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:37:39.005015 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2b1b8-e50f-42d5-84e4-87a5daa24868 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:39.008681 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48add404-260f-42c7-b85c-7a51a3e0f2d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:39.009292 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48add404-260f-42c7-b85c-7a51a3e0f2d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:39.009569 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48add404-260f-42c7-b85c-7a51a3e0f2d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:39.009839 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48add404-260f-42c7-b85c-7a51a3e0f2d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:39.019469 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48add404-260f-42c7-b85c-7a51a3e0f2d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:39.020938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48add404-260f-42c7-b85c-7a51a3e0f2d8 service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:39.030880 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1395/2750] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:37:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:37:40.301287 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb748cd-39f6-4eef-9c56-bea09fd36f5d None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:40.301925 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb748cd-39f6-4eef-9c56-bea09fd36f5d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:40.301925 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb748cd-39f6-4eef-9c56-bea09fd36f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:40.331270 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-5cb748cd-39f6-4eef-9c56-bea09fd36f5d None None] MFA Rules not processed for user `4552c0df2a3f4f30b6b2829de3a8284d`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:37:40.332618 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5cb748cd-39f6-4eef-9c56-bea09fd36f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:37:40.376365 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:37:40.399195 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1356/2751] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:37:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:37:40.410838 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48add404-260f-42c7-b85c-7a51a3e0f2d8 service neutron] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:40.412302 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-48add404-260f-42c7-b85c-7a51a3e0f2d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:37:40.470978 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48add404-260f-42c7-b85c-7a51a3e0f2d8 service neutron] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:40.474620 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e3251d-880f-4506-8182-0a87f3d98b53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:40.475217 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e3251d-880f-4506-8182-0a87f3d98b53 service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:40.475489 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e3251d-880f-4506-8182-0a87f3d98b53 service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:40.475754 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e3251d-880f-4506-8182-0a87f3d98b53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:40.497687 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e3251d-880f-4506-8182-0a87f3d98b53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:40.499355 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e3251d-880f-4506-8182-0a87f3d98b53 service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:40.508803 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1396/2752] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:37:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:37:43.343183 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13608d-598c-45e2-9a38-e44923393f2f None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:43.343998 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13608d-598c-45e2-9a38-e44923393f2f None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:43.344127 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13608d-598c-45e2-9a38-e44923393f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:43.363125 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-fa13608d-598c-45e2-9a38-e44923393f2f None None] MFA Rules not processed for user `4552c0df2a3f4f30b6b2829de3a8284d`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:37:43.364330 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-fa13608d-598c-45e2-9a38-e44923393f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:37:43.371848 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1357/2753] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:37:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:37:43.609091 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e3251d-880f-4506-8182-0a87f3d98b53 service cinder] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:43.610468 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-47e3251d-880f-4506-8182-0a87f3d98b53 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:37:43.622980 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e3251d-880f-4506-8182-0a87f3d98b53 service cinder] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:43.626476 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322fb8e4-01d7-4fe8-a886-d9f4296601e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:43.627063 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fb8e4-01d7-4fe8-a886-d9f4296601e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:43.627334 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fb8e4-01d7-4fe8-a886-d9f4296601e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:43.627599 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fb8e4-01d7-4fe8-a886-d9f4296601e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:43.629289 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-322fb8e4-01d7-4fe8-a886-d9f4296601e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:37:43.638337 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322fb8e4-01d7-4fe8-a886-d9f4296601e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:43.639731 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322fb8e4-01d7-4fe8-a886-d9f4296601e3 service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:43.647496 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1397/2754] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:37:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:37:53.358604 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=14 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:37:53.643966 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=65,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:37:54.441083 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa13608d-598c-45e2-9a38-e44923393f2f None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:37:54.442563 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-fa13608d-598c-45e2-9a38-e44923393f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:37:54.532246 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa13608d-598c-45e2-9a38-e44923393f2f None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:37:54.540014 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84084042-14c5-466d-943b-d3bd3c243a3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:37:54.540712 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84084042-14c5-466d-943b-d3bd3c243a3e service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:37:54.541049 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84084042-14c5-466d-943b-d3bd3c243a3e service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:37:54.541322 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84084042-14c5-466d-943b-d3bd3c243a3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:37:54.544124 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-84084042-14c5-466d-943b-d3bd3c243a3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:37:54.562015 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84084042-14c5-466d-943b-d3bd3c243a3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:37:54.563774 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84084042-14c5-466d-943b-d3bd3c243a3e service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:37:54.583053 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1358/2755] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:37:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:38:02.894987 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930b928-36a3-45af-95a4-5f3851bc353b None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:02.895462 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930b928-36a3-45af-95a4-5f3851bc353b None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:02.895462 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930b928-36a3-45af-95a4-5f3851bc353b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:02.914545 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-d930b928-36a3-45af-95a4-5f3851bc353b None None] MFA Rules not processed for user `4552c0df2a3f4f30b6b2829de3a8284d`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:38:02.915663 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-d930b928-36a3-45af-95a4-5f3851bc353b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:38:02.923181 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1398/2756] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:38:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:38:04.572433 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=34 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:38:12.909110 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=3 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:38:16.134844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5aa279-1ea0-4408-bcfe-9428d01d8384 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:16.135332 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5aa279-1ea0-4408-bcfe-9428d01d8384 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:16.135431 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5aa279-1ea0-4408-bcfe-9428d01d8384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:16.155749 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-2c5aa279-1ea0-4408-bcfe-9428d01d8384 None None] MFA Rules not processed for user `0cb8261b328c4cd8956762971fefceee`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:38:16.157589 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-2c5aa279-1ea0-4408-bcfe-9428d01d8384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:38:16.211943 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1359/2757] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:38:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:38:21.502387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692f9e0-c9a8-4927-b64b-496266d4bad8 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:21.502871 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692f9e0-c9a8-4927-b64b-496266d4bad8 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:21.502988 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692f9e0-c9a8-4927-b64b-496266d4bad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:21.581725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-6692f9e0-c9a8-4927-b64b-496266d4bad8 None None] MFA Rules not processed for user `17e3b509a2de4159bf234594573f5e18`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:38:21.583677 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-6692f9e0-c9a8-4927-b64b-496266d4bad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:38:21.683804 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1399/2758] 10.222.0.26 () {58 vars in 967 bytes} [Fri Aug 5 14:38:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:38:21.862740 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5aa279-1ea0-4408-bcfe-9428d01d8384 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:38:21.864339 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-2c5aa279-1ea0-4408-bcfe-9428d01d8384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:38:21.961176 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5aa279-1ea0-4408-bcfe-9428d01d8384 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:38:21.966722 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81635f2-a739-40ed-9a2e-9b9639c3f2ba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dddd7eea67f34f0cbb5f6a1cd85f1fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:38:21.967372 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81635f2-a739-40ed-9a2e-9b9639c3f2ba service swift] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:21.967655 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81635f2-a739-40ed-9a2e-9b9639c3f2ba service swift] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:21.967929 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81635f2-a739-40ed-9a2e-9b9639c3f2ba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:21.977903 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81635f2-a739-40ed-9a2e-9b9639c3f2ba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:38:21.979590 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81635f2-a739-40ed-9a2e-9b9639c3f2ba service swift] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:38:21.989079 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1360/2759] 10.222.0.26 () {60 vars in 1384 bytes} [Fri Aug 5 14:38:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 14:38:26.997073 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692f9e0-c9a8-4927-b64b-496266d4bad8 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:38:26.999927 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-6692f9e0-c9a8-4927-b64b-496266d4bad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:38:27.096781 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692f9e0-c9a8-4927-b64b-496266d4bad8 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:38:27.105142 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2168d7b-8fe5-4291-b78c-972db9256b96 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff761f6f81a46098f94bd2ec2be9c1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:38:27.105712 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2168d7b-8fe5-4291-b78c-972db9256b96 service placement] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:27.105985 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2168d7b-8fe5-4291-b78c-972db9256b96 service placement] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:27.106247 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2168d7b-8fe5-4291-b78c-972db9256b96 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:27.108622 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-a2168d7b-8fe5-4291-b78c-972db9256b96 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:38:27.122336 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2168d7b-8fe5-4291-b78c-972db9256b96 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:38:27.123919 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2168d7b-8fe5-4291-b78c-972db9256b96 service placement] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:38:27.135632 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1400/2760] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:38:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:38:31.121318 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81635f2-a739-40ed-9a2e-9b9639c3f2ba service swift] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:38:31.122800 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e81635f2-a739-40ed-9a2e-9b9639c3f2ba service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:38:31.137129 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81635f2-a739-40ed-9a2e-9b9639c3f2ba service swift] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:38:31.142134 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d511afd3-3c5b-486a-8a89-dc02d1270e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:38:31.142768 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511afd3-3c5b-486a-8a89-dc02d1270e49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:31.143041 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511afd3-3c5b-486a-8a89-dc02d1270e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:31.143304 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511afd3-3c5b-486a-8a89-dc02d1270e49 admin admin] PATH_INFO: `/v3/users/7d37124881084b16a73f5eed20c70940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:31.144464 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d511afd3-3c5b-486a-8a89-dc02d1270e49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d37124881084b16a73f5eed20c70940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:38:31.146358 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d511afd3-3c5b-486a-8a89-dc02d1270e49 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:38:31.328611 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-d511afd3-3c5b-486a-8a89-dc02d1270e49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d37124881084b16a73f5eed20c70940'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:38:31.340503 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-d511afd3-3c5b-486a-8a89-dc02d1270e49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d37124881084b16a73f5eed20c70940'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:38:31.344515 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:38:31.419541 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-d511afd3-3c5b-486a-8a89-dc02d1270e49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d37124881084b16a73f5eed20c70940'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:38:31.420279 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-d511afd3-3c5b-486a-8a89-dc02d1270e49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d37124881084b16a73f5eed20c70940'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:38:31.428761 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-d511afd3-3c5b-486a-8a89-dc02d1270e49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d37124881084b16a73f5eed20c70940'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:38:31.435675 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1361/2761] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:38:31 2022] DELETE /identity/v3/users/7d37124881084b16a73f5eed20c70940 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:38:31.689780 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2168d7b-8fe5-4291-b78c-972db9256b96 service placement] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:38:31.702457 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2168d7b-8fe5-4291-b78c-972db9256b96 service placement] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:38:31.763897 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75b195-ff32-414d-b759-6b35ac8944d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:38:31.764545 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75b195-ff32-414d-b759-6b35ac8944d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:31.765370 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75b195-ff32-414d-b759-6b35ac8944d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:31.765724 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75b195-ff32-414d-b759-6b35ac8944d1 admin admin] PATH_INFO: `/v3/projects/ca094dd6e4d64383b75640f040744140` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:31.767254 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f75b195-ff32-414d-b759-6b35ac8944d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca094dd6e4d64383b75640f040744140)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:38:31.769243 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f75b195-ff32-414d-b759-6b35ac8944d1 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:38:31.772060 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:38:31.860401 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-5f75b195-ff32-414d-b759-6b35ac8944d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca094dd6e4d64383b75640f040744140', 'request_id': 'req-5f75b195-ff32-414d-b759-6b35ac8944d1'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:38:31.867471 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-5f75b195-ff32-414d-b759-6b35ac8944d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca094dd6e4d64383b75640f040744140', 'request_id': 'req-5f75b195-ff32-414d-b759-6b35ac8944d1'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:38:31.873247 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-5f75b195-ff32-414d-b759-6b35ac8944d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca094dd6e4d64383b75640f040744140', 'request_id': 'req-5f75b195-ff32-414d-b759-6b35ac8944d1'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:38:31.939727 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-5f75b195-ff32-414d-b759-6b35ac8944d1 admin admin] The token cache is being invalidate because project ca094dd6e4d64383b75640f040744140 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:38:31.940152 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-5f75b195-ff32-414d-b759-6b35ac8944d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:38:31.941049 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-5f75b195-ff32-414d-b759-6b35ac8944d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:38:31.943201 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1401/2762] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:38:31 2022] DELETE /identity/v3/projects/ca094dd6e4d64383b75640f040744140 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:38:31.947243 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d511afd3-3c5b-486a-8a89-dc02d1270e49 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:38:31.948714 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d511afd3-3c5b-486a-8a89-dc02d1270e49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:38:31.962459 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d511afd3-3c5b-486a-8a89-dc02d1270e49 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:38:32.026723 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442d4f86-5aae-49f7-a26b-917462f0e33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:38:32.027360 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d4f86-5aae-49f7-a26b-917462f0e33a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:32.027644 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d4f86-5aae-49f7-a26b-917462f0e33a admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:32.027909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d4f86-5aae-49f7-a26b-917462f0e33a admin admin] PATH_INFO: `/v3/users/9457d9aede4247cc93c5a7aabc7984ec` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:32.029096 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442d4f86-5aae-49f7-a26b-917462f0e33a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9457d9aede4247cc93c5a7aabc7984ec)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:38:32.031144 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442d4f86-5aae-49f7-a26b-917462f0e33a admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:38:32.186096 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-442d4f86-5aae-49f7-a26b-917462f0e33a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9457d9aede4247cc93c5a7aabc7984ec'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:38:32.197753 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-442d4f86-5aae-49f7-a26b-917462f0e33a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9457d9aede4247cc93c5a7aabc7984ec'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:38:32.253484 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-442d4f86-5aae-49f7-a26b-917462f0e33a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9457d9aede4247cc93c5a7aabc7984ec'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:38:32.254085 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-442d4f86-5aae-49f7-a26b-917462f0e33a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9457d9aede4247cc93c5a7aabc7984ec'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:38:32.261397 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-442d4f86-5aae-49f7-a26b-917462f0e33a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9457d9aede4247cc93c5a7aabc7984ec'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:38:32.267738 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1362/2763] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:38:31 2022] DELETE /identity/v3/users/9457d9aede4247cc93c5a7aabc7984ec => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:38:32.508787 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75b195-ff32-414d-b759-6b35ac8944d1 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:38:32.510168 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-5f75b195-ff32-414d-b759-6b35ac8944d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:38:32.541527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75b195-ff32-414d-b759-6b35ac8944d1 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:38:32.609743 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b3d900-28a6-478a-9c05-6076a2b98206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:38:32.610700 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3d900-28a6-478a-9c05-6076a2b98206 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:32.611147 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3d900-28a6-478a-9c05-6076a2b98206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:32.611573 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3d900-28a6-478a-9c05-6076a2b98206 admin admin] PATH_INFO: `/v3/projects/051c21501570451e966a6cd64dc5b7be` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:32.613040 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b3d900-28a6-478a-9c05-6076a2b98206 admin admin] RBAC: Authorizing `identity:delete_project(project_id=051c21501570451e966a6cd64dc5b7be)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:38:32.615142 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b3d900-28a6-478a-9c05-6076a2b98206 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:38:32.722492 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-69b3d900-28a6-478a-9c05-6076a2b98206 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '051c21501570451e966a6cd64dc5b7be', 'request_id': 'req-69b3d900-28a6-478a-9c05-6076a2b98206'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:38:32.729210 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-69b3d900-28a6-478a-9c05-6076a2b98206 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '051c21501570451e966a6cd64dc5b7be', 'request_id': 'req-69b3d900-28a6-478a-9c05-6076a2b98206'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:38:32.734903 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-69b3d900-28a6-478a-9c05-6076a2b98206 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '051c21501570451e966a6cd64dc5b7be', 'request_id': 'req-69b3d900-28a6-478a-9c05-6076a2b98206'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:38:32.813750 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-69b3d900-28a6-478a-9c05-6076a2b98206 admin admin] The token cache is being invalidate because project 051c21501570451e966a6cd64dc5b7be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:38:32.814037 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-69b3d900-28a6-478a-9c05-6076a2b98206 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:38:32.814776 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-69b3d900-28a6-478a-9c05-6076a2b98206 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:38:32.816204 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1402/2764] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:38:32 2022] DELETE /identity/v3/projects/051c21501570451e966a6cd64dc5b7be => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:38:32.822444 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafd884-eaa9-4b0b-af13-1f212bcc2325 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:32.822990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafd884-eaa9-4b0b-af13-1f212bcc2325 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:32.823441 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafd884-eaa9-4b0b-af13-1f212bcc2325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:32.824799 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa71746-a7a6-4797-bcca-adca52eca46d None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:32.826872 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa71746-a7a6-4797-bcca-adca52eca46d None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:32.826872 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa71746-a7a6-4797-bcca-adca52eca46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:32.846199 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-0aa71746-a7a6-4797-bcca-adca52eca46d None None] MFA Rules not processed for user `17e3b509a2de4159bf234594573f5e18`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:38:32.852096 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-ccafd884-eaa9-4b0b-af13-1f212bcc2325 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:38:32.852384 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-0aa71746-a7a6-4797-bcca-adca52eca46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:38:32.852529 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ccafd884-eaa9-4b0b-af13-1f212bcc2325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:38:32.864879 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:38:32.904924 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1403/2765] 10.222.0.26 () {58 vars in 1009 bytes} [Fri Aug 5 14:38:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:38:32.909900 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1363/2766] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:38:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:38:32.917882 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db07b38-bc22-4085-89a7-634c417b4879 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:32.918158 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccafd884-eaa9-4b0b-af13-1f212bcc2325 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:38:32.918296 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db07b38-bc22-4085-89a7-634c417b4879 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:32.918434 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db07b38-bc22-4085-89a7-634c417b4879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:32.919504 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ccafd884-eaa9-4b0b-af13-1f212bcc2325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:38:32.930259 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:38:32.941535 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-0db07b38-bc22-4085-89a7-634c417b4879 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:38:32.942910 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-0db07b38-bc22-4085-89a7-634c417b4879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:38:32.951336 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1404/2767] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:38:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:38:32.955644 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db07b38-bc22-4085-89a7-634c417b4879 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:38:32.964443 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db07b38-bc22-4085-89a7-634c417b4879 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:38:32.969867 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc5d2d-decc-4441-925b-4efd54555f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:38:32.970985 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc5d2d-decc-4441-925b-4efd54555f99 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:32.970985 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc5d2d-decc-4441-925b-4efd54555f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:32.970985 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc5d2d-decc-4441-925b-4efd54555f99 admin admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:32.971642 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecc5d2d-decc-4441-925b-4efd54555f99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:38:32.971944 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecc5d2d-decc-4441-925b-4efd54555f99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:38:32.973548 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecc5d2d-decc-4441-925b-4efd54555f99 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:38:32.986068 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1405/2768] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:38:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:38:32.990461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc5d2d-decc-4441-925b-4efd54555f99 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:38:32.995748 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccafd884-eaa9-4b0b-af13-1f212bcc2325 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:38:33.001229 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc5d2d-decc-4441-925b-4efd54555f99 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:38:33.001494 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90832218-8beb-4602-b612-65e8a3eda19f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dddd7eea67f34f0cbb5f6a1cd85f1fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:38:33.002112 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90832218-8beb-4602-b612-65e8a3eda19f service swift] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:33.002386 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90832218-8beb-4602-b612-65e8a3eda19f service swift] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:33.002652 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90832218-8beb-4602-b612-65e8a3eda19f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:33.005906 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca92a7f-eb4d-4ee6-9994-7f53663309ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:38:33.006499 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca92a7f-eb4d-4ee6-9994-7f53663309ea admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:33.006779 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca92a7f-eb4d-4ee6-9994-7f53663309ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:33.007051 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca92a7f-eb4d-4ee6-9994-7f53663309ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:33.007707 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca92a7f-eb4d-4ee6-9994-7f53663309ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:38:33.009631 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca92a7f-eb4d-4ee6-9994-7f53663309ea admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:38:33.010695 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90832218-8beb-4602-b612-65e8a3eda19f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:38:33.012254 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90832218-8beb-4602-b612-65e8a3eda19f service swift] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:38:33.021260 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1364/2769] 10.222.0.26 () {60 vars in 1384 bytes} [Fri Aug 5 14:38:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 05 14:38:33.096288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1406/2770] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:38:32 2022] POST /identity/v3/projects => generated 379 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:38:33.100405 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90832218-8beb-4602-b612-65e8a3eda19f service swift] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:38:33.111729 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90832218-8beb-4602-b612-65e8a3eda19f service swift] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:38:33.180984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e876ee-f179-4a95-a3f2-407d6cbc721b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:38:33.181606 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e876ee-f179-4a95-a3f2-407d6cbc721b admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:33.181885 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e876ee-f179-4a95-a3f2-407d6cbc721b admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:33.182153 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e876ee-f179-4a95-a3f2-407d6cbc721b admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:33.182837 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e876ee-f179-4a95-a3f2-407d6cbc721b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:38:33.184756 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e876ee-f179-4a95-a3f2-407d6cbc721b admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:38:33.300762 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1365/2771] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:38:33 2022] POST /identity/v3/users => generated 290 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:38:33.304944 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca92a7f-eb4d-4ee6-9994-7f53663309ea admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:38:33.319039 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca92a7f-eb4d-4ee6-9994-7f53663309ea admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:38:33.325044 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff917eb2-b4fd-48dc-9441-81571c354ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:38:33.325663 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff917eb2-b4fd-48dc-9441-81571c354ca0 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:33.325938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff917eb2-b4fd-48dc-9441-81571c354ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:33.326285 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff917eb2-b4fd-48dc-9441-81571c354ca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:33.326955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff917eb2-b4fd-48dc-9441-81571c354ca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:38:33.328643 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff917eb2-b4fd-48dc-9441-81571c354ca0 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:38:33.348552 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1407/2772] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:38:33.352147 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e876ee-f179-4a95-a3f2-407d6cbc721b admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:38:33.367812 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e876ee-f179-4a95-a3f2-407d6cbc721b admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:38:33.373288 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc52ce7-7180-445e-9073-25c44c8ec4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:38:33.373896 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc52ce7-7180-445e-9073-25c44c8ec4f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:33.374214 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc52ce7-7180-445e-9073-25c44c8ec4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:33.375106 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc52ce7-7180-445e-9073-25c44c8ec4f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:33.375816 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc52ce7-7180-445e-9073-25c44c8ec4f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:38:33.377439 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc52ce7-7180-445e-9073-25c44c8ec4f4 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:38:33.389733 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1366/2773] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:38:33.394414 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff917eb2-b4fd-48dc-9441-81571c354ca0 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:38:33.404751 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff917eb2-b4fd-48dc-9441-81571c354ca0 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:38:33.410372 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfcf959-504f-4f47-b47f-a8101db4f6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:38:33.411539 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcf959-504f-4f47-b47f-a8101db4f6c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:33.411539 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcf959-504f-4f47-b47f-a8101db4f6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:33.411539 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcf959-504f-4f47-b47f-a8101db4f6c9 admin admin] PATH_INFO: `/v3/projects/d408cabccaa44c338d6031955d3a1f7c/users/f7b9ccd1b8cc4832a9c77513b5f21815/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:33.428383 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfcf959-504f-4f47-b47f-a8101db4f6c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d408cabccaa44c338d6031955d3a1f7c, user_id=f7b9ccd1b8cc4832a9c77513b5f21815, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:38:33.431425 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfcf959-504f-4f47-b47f-a8101db4f6c9 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:38:33.535092 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1408/2774] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:38:33 2022] PUT /identity/v3/projects/d408cabccaa44c338d6031955d3a1f7c/users/f7b9ccd1b8cc4832a9c77513b5f21815/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:38:33.540360 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec977a-6b01-4a4f-92df-4cf5678d1c82 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:33.540941 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec977a-6b01-4a4f-92df-4cf5678d1c82 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:33.541444 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec977a-6b01-4a4f-92df-4cf5678d1c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:33.568109 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-f6ec977a-6b01-4a4f-92df-4cf5678d1c82 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:38:33.569913 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f6ec977a-6b01-4a4f-92df-4cf5678d1c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:38:33.642132 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1367/2775] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:38:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:38:33.652428 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfcf959-504f-4f47-b47f-a8101db4f6c9 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:38:33.654714 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-6dfcf959-504f-4f47-b47f-a8101db4f6c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:38:33.731392 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfcf959-504f-4f47-b47f-a8101db4f6c9 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:38:33.737095 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9700c7-1f0b-4402-b339-64cd1af3c40b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:38:33.737748 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9700c7-1f0b-4402-b339-64cd1af3c40b service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:33.738024 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9700c7-1f0b-4402-b339-64cd1af3c40b service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:33.738295 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9700c7-1f0b-4402-b339-64cd1af3c40b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:33.750069 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9700c7-1f0b-4402-b339-64cd1af3c40b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:38:33.751909 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9700c7-1f0b-4402-b339-64cd1af3c40b service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:38:33.763571 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1409/2776] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:38:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:38:41.094297 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fe3ad-7005-4ede-b0f5-4dbaa1f3d95d None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:41.095146 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fe3ad-7005-4ede-b0f5-4dbaa1f3d95d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:41.095573 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fe3ad-7005-4ede-b0f5-4dbaa1f3d95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:41.132378 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-8a6fe3ad-7005-4ede-b0f5-4dbaa1f3d95d None None] MFA Rules not processed for user `f7b9ccd1b8cc4832a9c77513b5f21815`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:38:41.134906 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8a6fe3ad-7005-4ede-b0f5-4dbaa1f3d95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:38:41.139842 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:38:41.229713 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1368/2777] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:38:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:38:41.234706 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9700c7-1f0b-4402-b339-64cd1af3c40b service neutron] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:38:41.248649 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9700c7-1f0b-4402-b339-64cd1af3c40b service neutron] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:38:41.253772 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33d1671-c9a4-440e-abb9-cdf6ce56cbee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:38:41.254652 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d1671-c9a4-440e-abb9-cdf6ce56cbee admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:41.255111 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d1671-c9a4-440e-abb9-cdf6ce56cbee admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:41.255380 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d1671-c9a4-440e-abb9-cdf6ce56cbee admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:41.256181 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33d1671-c9a4-440e-abb9-cdf6ce56cbee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:38:41.258976 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33d1671-c9a4-440e-abb9-cdf6ce56cbee admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:38:41.341223 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1410/2778] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:38:41 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:38:41.346284 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6fe3ad-7005-4ede-b0f5-4dbaa1f3d95d None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:38:41.360338 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6fe3ad-7005-4ede-b0f5-4dbaa1f3d95d None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:38:41.435308 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708433e7-e04d-4c67-8f2d-8c3ec01d76ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:38:41.435946 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708433e7-e04d-4c67-8f2d-8c3ec01d76ec admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:41.436222 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708433e7-e04d-4c67-8f2d-8c3ec01d76ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:41.437087 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708433e7-e04d-4c67-8f2d-8c3ec01d76ec admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:41.437841 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708433e7-e04d-4c67-8f2d-8c3ec01d76ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:38:41.439763 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708433e7-e04d-4c67-8f2d-8c3ec01d76ec admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:38:41.534750 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1369/2779] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:38:41 2022] POST /identity/v3/users => generated 295 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:38:41.538830 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33d1671-c9a4-440e-abb9-cdf6ce56cbee admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:38:41.552928 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33d1671-c9a4-440e-abb9-cdf6ce56cbee admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:38:41.563535 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0f172-a0c0-4cd5-b24b-6391022b1f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:38:41.564769 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0f172-a0c0-4cd5-b24b-6391022b1f8d admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:41.565338 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0f172-a0c0-4cd5-b24b-6391022b1f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:41.566437 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0f172-a0c0-4cd5-b24b-6391022b1f8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:41.567766 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf0f172-a0c0-4cd5-b24b-6391022b1f8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:38:41.570077 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf0f172-a0c0-4cd5-b24b-6391022b1f8d admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:38:41.589216 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1411/2780] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:38:41.593623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708433e7-e04d-4c67-8f2d-8c3ec01d76ec admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:38:41.607710 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708433e7-e04d-4c67-8f2d-8c3ec01d76ec admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:38:41.615624 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab255e8b-3997-447d-8a02-42570c79e3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:38:41.616586 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab255e8b-3997-447d-8a02-42570c79e3db admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:41.616849 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab255e8b-3997-447d-8a02-42570c79e3db admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:41.617343 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab255e8b-3997-447d-8a02-42570c79e3db admin admin] PATH_INFO: `/v3/projects/da68239ae0ec4ebb8e3962bd067f4097/users/d26da9b0c19544cd95bb818b71225174/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:41.638531 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab255e8b-3997-447d-8a02-42570c79e3db admin admin] RBAC: Authorizing `identity:create_grant(project_id=da68239ae0ec4ebb8e3962bd067f4097, user_id=d26da9b0c19544cd95bb818b71225174, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:38:41.641854 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab255e8b-3997-447d-8a02-42570c79e3db admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:38:41.758740 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1370/2781] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:38:41 2022] PUT /identity/v3/projects/da68239ae0ec4ebb8e3962bd067f4097/users/d26da9b0c19544cd95bb818b71225174/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:38:41.763075 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0f172-a0c0-4cd5-b24b-6391022b1f8d admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:38:41.776215 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0f172-a0c0-4cd5-b24b-6391022b1f8d admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:38:41.804379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:38:41.863639 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d140a0-a475-4add-9502-c4f1a0ee913f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:38:41.864289 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d140a0-a475-4add-9502-c4f1a0ee913f admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:41.864938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d140a0-a475-4add-9502-c4f1a0ee913f admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:41.864938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d140a0-a475-4add-9502-c4f1a0ee913f admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:41.865571 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d140a0-a475-4add-9502-c4f1a0ee913f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:38:41.867168 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d140a0-a475-4add-9502-c4f1a0ee913f admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:38:41.881533 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1412/2782] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:38:41.885333 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab255e8b-3997-447d-8a02-42570c79e3db admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:38:41.898109 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab255e8b-3997-447d-8a02-42570c79e3db admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:38:41.902924 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce50ce7d-b07b-4fb2-8809-d424db6b4ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:38:41.903535 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50ce7d-b07b-4fb2-8809-d424db6b4ec8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:41.903809 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50ce7d-b07b-4fb2-8809-d424db6b4ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:41.904079 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50ce7d-b07b-4fb2-8809-d424db6b4ec8 admin admin] PATH_INFO: `/v3/projects/da68239ae0ec4ebb8e3962bd067f4097/users/d26da9b0c19544cd95bb818b71225174/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:41.906839 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce50ce7d-b07b-4fb2-8809-d424db6b4ec8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da68239ae0ec4ebb8e3962bd067f4097, user_id=d26da9b0c19544cd95bb818b71225174, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:38:41.909540 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce50ce7d-b07b-4fb2-8809-d424db6b4ec8 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:38:41.922435 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1371/2783] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:38:41 2022] PUT /identity/v3/projects/da68239ae0ec4ebb8e3962bd067f4097/users/d26da9b0c19544cd95bb818b71225174/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:38:48.466078 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4ee6f-21a1-4a83-bc2f-4a7fc9af854c None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:48.466573 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4ee6f-21a1-4a83-bc2f-4a7fc9af854c None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:48.466802 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4ee6f-21a1-4a83-bc2f-4a7fc9af854c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:48.498194 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-a8c4ee6f-21a1-4a83-bc2f-4a7fc9af854c None None] MFA Rules not processed for user `d26da9b0c19544cd95bb818b71225174`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:38:48.499490 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-a8c4ee6f-21a1-4a83-bc2f-4a7fc9af854c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:38:48.618274 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1413/2784] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:38:48 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:38:48.635595 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce50ce7d-b07b-4fb2-8809-d424db6b4ec8 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:38:48.643679 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ce50ce7d-b07b-4fb2-8809-d424db6b4ec8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:38:48.703013 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:38:48.726169 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce50ce7d-b07b-4fb2-8809-d424db6b4ec8 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:38:48.731937 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3e146-9f82-48ea-a116-37481cff3459 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:38:48.732748 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3e146-9f82-48ea-a116-37481cff3459 service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:48.733149 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3e146-9f82-48ea-a116-37481cff3459 service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:48.735467 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3e146-9f82-48ea-a116-37481cff3459 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:48.834117 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd3e146-9f82-48ea-a116-37481cff3459 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:38:48.835715 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd3e146-9f82-48ea-a116-37481cff3459 service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:38:48.860677 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1372/2785] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:38:48 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:38:54.715770 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c6279-93fe-4a53-8876-13a4306ed14b None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:54.716709 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c6279-93fe-4a53-8876-13a4306ed14b None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:54.717061 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c6279-93fe-4a53-8876-13a4306ed14b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:54.738640 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-fb1c6279-93fe-4a53-8876-13a4306ed14b None None] MFA Rules not processed for user `17e3b509a2de4159bf234594573f5e18`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:38:54.739911 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-fb1c6279-93fe-4a53-8876-13a4306ed14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:38:54.806368 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1414/2786] 10.222.0.26 () {58 vars in 967 bytes} [Fri Aug 5 14:38:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:38:54.823734 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3e146-9f82-48ea-a116-37481cff3459 service nova] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:38:54.901527 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3e146-9f82-48ea-a116-37481cff3459 service nova] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:38:54.907332 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6267e277-16cf-4b2b-ac55-ae3cf46c074a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dddd7eea67f34f0cbb5f6a1cd85f1fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:38:54.907970 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267e277-16cf-4b2b-ac55-ae3cf46c074a service swift] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:54.908249 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267e277-16cf-4b2b-ac55-ae3cf46c074a service swift] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:54.908555 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267e277-16cf-4b2b-ac55-ae3cf46c074a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:54.917846 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6267e277-16cf-4b2b-ac55-ae3cf46c074a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:38:54.923268 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6267e277-16cf-4b2b-ac55-ae3cf46c074a service swift] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:38:54.929831 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1373/2787] 10.222.0.26 () {60 vars in 1384 bytes} [Fri Aug 5 14:38:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 14:38:58.157953 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c6279-93fe-4a53-8876-13a4306ed14b None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:38:58.159424 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-fb1c6279-93fe-4a53-8876-13a4306ed14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:38:58.214042 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:38:58.231782 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c6279-93fe-4a53-8876-13a4306ed14b None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:38:58.236327 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc1a50-39df-4865-8c15-fb4acb54d11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:38:58.236993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc1a50-39df-4865-8c15-fb4acb54d11f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:58.237275 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc1a50-39df-4865-8c15-fb4acb54d11f admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:58.237552 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc1a50-39df-4865-8c15-fb4acb54d11f admin admin] PATH_INFO: `/v3/users/4552c0df2a3f4f30b6b2829de3a8284d` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:58.238735 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dc1a50-39df-4865-8c15-fb4acb54d11f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4552c0df2a3f4f30b6b2829de3a8284d)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:38:58.240790 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dc1a50-39df-4865-8c15-fb4acb54d11f admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:38:58.395858 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-c7dc1a50-39df-4865-8c15-fb4acb54d11f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4552c0df2a3f4f30b6b2829de3a8284d'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:38:58.403205 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-c7dc1a50-39df-4865-8c15-fb4acb54d11f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4552c0df2a3f4f30b6b2829de3a8284d'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:38:58.403851 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-c7dc1a50-39df-4865-8c15-fb4acb54d11f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4552c0df2a3f4f30b6b2829de3a8284d'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:38:58.404469 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-c7dc1a50-39df-4865-8c15-fb4acb54d11f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4552c0df2a3f4f30b6b2829de3a8284d'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:38:58.419906 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-c7dc1a50-39df-4865-8c15-fb4acb54d11f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4552c0df2a3f4f30b6b2829de3a8284d'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:38:58.464509 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1415/2788] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:38:58 2022] DELETE /identity/v3/users/4552c0df2a3f4f30b6b2829de3a8284d => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:38:58.711544 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6267e277-16cf-4b2b-ac55-ae3cf46c074a service swift] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:38:58.723322 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6267e277-16cf-4b2b-ac55-ae3cf46c074a service swift] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:38:58.787198 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04826d19-f545-4ec8-9cab-338e3643ad53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:38:58.787772 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04826d19-f545-4ec8-9cab-338e3643ad53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:58.788050 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04826d19-f545-4ec8-9cab-338e3643ad53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:58.788322 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04826d19-f545-4ec8-9cab-338e3643ad53 admin admin] PATH_INFO: `/v3/projects/58c88a9de1de4e2b83bb8a37142608a6` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:58.789528 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04826d19-f545-4ec8-9cab-338e3643ad53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58c88a9de1de4e2b83bb8a37142608a6)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:38:58.791430 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04826d19-f545-4ec8-9cab-338e3643ad53 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:38:58.888142 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:38:58.907171 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-04826d19-f545-4ec8-9cab-338e3643ad53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58c88a9de1de4e2b83bb8a37142608a6', 'request_id': 'req-04826d19-f545-4ec8-9cab-338e3643ad53'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:38:58.913555 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-04826d19-f545-4ec8-9cab-338e3643ad53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58c88a9de1de4e2b83bb8a37142608a6', 'request_id': 'req-04826d19-f545-4ec8-9cab-338e3643ad53'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:38:59.002326 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-04826d19-f545-4ec8-9cab-338e3643ad53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58c88a9de1de4e2b83bb8a37142608a6', 'request_id': 'req-04826d19-f545-4ec8-9cab-338e3643ad53'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:38:59.013577 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-04826d19-f545-4ec8-9cab-338e3643ad53 admin admin] The token cache is being invalidate because project 58c88a9de1de4e2b83bb8a37142608a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:38:59.013862 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-04826d19-f545-4ec8-9cab-338e3643ad53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:38:59.014490 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-04826d19-f545-4ec8-9cab-338e3643ad53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:38:59.016742 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1374/2789] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:38:58 2022] DELETE /identity/v3/projects/58c88a9de1de4e2b83bb8a37142608a6 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:38:59.021793 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376d4bd-d2f6-4cd9-aa87-e29eb6a852d3 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:59.022026 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376d4bd-d2f6-4cd9-aa87-e29eb6a852d3 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:59.022365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376d4bd-d2f6-4cd9-aa87-e29eb6a852d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:59.043962 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-9376d4bd-d2f6-4cd9-aa87-e29eb6a852d3 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:38:59.045125 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-9376d4bd-d2f6-4cd9-aa87-e29eb6a852d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:38:59.105425 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1416/2790] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:38:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:38:59.113379 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ee1ac-87d2-47c9-ac44-7d611f27bc7f None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:59.113695 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ee1ac-87d2-47c9-ac44-7d611f27bc7f None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:59.113936 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ee1ac-87d2-47c9-ac44-7d611f27bc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:59.135331 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-f66ee1ac-87d2-47c9-ac44-7d611f27bc7f None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:38:59.136574 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f66ee1ac-87d2-47c9-ac44-7d611f27bc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:38:59.145249 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1375/2791] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:38:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:38:59.149541 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9376d4bd-d2f6-4cd9-aa87-e29eb6a852d3 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:38:59.159052 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9376d4bd-d2f6-4cd9-aa87-e29eb6a852d3 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:38:59.163574 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5659f920-ad84-46d4-92a1-278ded6f47e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:38:59.164218 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659f920-ad84-46d4-92a1-278ded6f47e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:59.164747 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659f920-ad84-46d4-92a1-278ded6f47e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:59.164973 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659f920-ad84-46d4-92a1-278ded6f47e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:59.165485 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5659f920-ad84-46d4-92a1-278ded6f47e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:38:59.165781 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5659f920-ad84-46d4-92a1-278ded6f47e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:38:59.167388 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5659f920-ad84-46d4-92a1-278ded6f47e3 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:38:59.179739 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1417/2792] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:38:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:38:59.184274 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ee1ac-87d2-47c9-ac44-7d611f27bc7f None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:38:59.194963 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ee1ac-87d2-47c9-ac44-7d611f27bc7f None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:38:59.199694 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e77102-b957-432d-8e36-96c23cd83971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:38:59.200328 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e77102-b957-432d-8e36-96c23cd83971 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:59.200635 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e77102-b957-432d-8e36-96c23cd83971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:59.200929 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e77102-b957-432d-8e36-96c23cd83971 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:59.201577 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e77102-b957-432d-8e36-96c23cd83971 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:38:59.204111 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e77102-b957-432d-8e36-96c23cd83971 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:38:59.289824 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1376/2793] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:38:59 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:38:59.294109 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5659f920-ad84-46d4-92a1-278ded6f47e3 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:38:59.304047 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5659f920-ad84-46d4-92a1-278ded6f47e3 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:38:59.351910 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:38:59.364644 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e49f2e-6214-4c62-8999-9ea2c1eb7b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:38:59.365279 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e49f2e-6214-4c62-8999-9ea2c1eb7b75 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:59.370528 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e49f2e-6214-4c62-8999-9ea2c1eb7b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:59.370528 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e49f2e-6214-4c62-8999-9ea2c1eb7b75 admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:59.370528 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e49f2e-6214-4c62-8999-9ea2c1eb7b75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:38:59.370528 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e49f2e-6214-4c62-8999-9ea2c1eb7b75 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:38:59.476979 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1418/2794] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:38:59 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:38:59.481487 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e77102-b957-432d-8e36-96c23cd83971 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:38:59.494525 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e77102-b957-432d-8e36-96c23cd83971 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:38:59.501306 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac6328-90c6-44c4-8c94-662040dcc4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:38:59.502108 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac6328-90c6-44c4-8c94-662040dcc4fc admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:59.502388 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac6328-90c6-44c4-8c94-662040dcc4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:59.502748 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac6328-90c6-44c4-8c94-662040dcc4fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:59.503491 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ac6328-90c6-44c4-8c94-662040dcc4fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:38:59.505460 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ac6328-90c6-44c4-8c94-662040dcc4fc admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:38:59.520637 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1377/2795] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:38:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:38:59.524903 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e49f2e-6214-4c62-8999-9ea2c1eb7b75 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:38:59.536109 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e49f2e-6214-4c62-8999-9ea2c1eb7b75 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:38:59.541472 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0cfae1-0219-44bb-88f8-4ac52db38e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:38:59.542105 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0cfae1-0219-44bb-88f8-4ac52db38e59 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:59.542385 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0cfae1-0219-44bb-88f8-4ac52db38e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:59.542795 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0cfae1-0219-44bb-88f8-4ac52db38e59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:59.543365 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0cfae1-0219-44bb-88f8-4ac52db38e59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:38:59.545154 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0cfae1-0219-44bb-88f8-4ac52db38e59 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:38:59.558147 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1419/2796] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:38:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:38:59.563782 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac6328-90c6-44c4-8c94-662040dcc4fc admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:38:59.574607 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac6328-90c6-44c4-8c94-662040dcc4fc admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:38:59.579480 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a805b2-d1a6-4e5b-bc2b-1fd2f9bba217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:38:59.580164 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a805b2-d1a6-4e5b-bc2b-1fd2f9bba217 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:59.580468 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a805b2-d1a6-4e5b-bc2b-1fd2f9bba217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:59.580749 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a805b2-d1a6-4e5b-bc2b-1fd2f9bba217 admin admin] PATH_INFO: `/v3/projects/c442c235b363456eb569a027799d949d/users/34e4fe82fb76414dbd7ccfe8ff5b68b2/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:59.597791 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a805b2-d1a6-4e5b-bc2b-1fd2f9bba217 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c442c235b363456eb569a027799d949d, user_id=34e4fe82fb76414dbd7ccfe8ff5b68b2, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:38:59.600565 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a805b2-d1a6-4e5b-bc2b-1fd2f9bba217 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:38:59.773123 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1378/2797] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:38:59 2022] PUT /identity/v3/projects/c442c235b363456eb569a027799d949d/users/34e4fe82fb76414dbd7ccfe8ff5b68b2/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:38:59.784219 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c567664-9134-44cc-bdaf-ef0c3d4bd055 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:59.787287 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c567664-9134-44cc-bdaf-ef0c3d4bd055 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:59.787287 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c567664-9134-44cc-bdaf-ef0c3d4bd055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:59.815142 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-4c567664-9134-44cc-bdaf-ef0c3d4bd055 None None] MFA Rules not processed for user `34e4fe82fb76414dbd7ccfe8ff5b68b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:38:59.816296 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-4c567664-9134-44cc-bdaf-ef0c3d4bd055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:38:59.881016 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1420/2798] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:38:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:38:59.892563 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a805b2-d1a6-4e5b-bc2b-1fd2f9bba217 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:38:59.894243 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-b4a805b2-d1a6-4e5b-bc2b-1fd2f9bba217 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:38:59.956623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a805b2-d1a6-4e5b-bc2b-1fd2f9bba217 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:38:59.960241 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec68334-1c9e-4235-80f4-582a79920f83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:38:59.960885 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec68334-1c9e-4235-80f4-582a79920f83 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:38:59.961157 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec68334-1c9e-4235-80f4-582a79920f83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:38:59.961425 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec68334-1c9e-4235-80f4-582a79920f83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:38:59.970853 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec68334-1c9e-4235-80f4-582a79920f83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:38:59.972259 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec68334-1c9e-4235-80f4-582a79920f83 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:38:59.981041 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1379/2799] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:38:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:39:00.008121 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c567664-9134-44cc-bdaf-ef0c3d4bd055 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:39:00.077845 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c567664-9134-44cc-bdaf-ef0c3d4bd055 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:39:00.082663 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f96a32-894a-4494-aba7-de90ce0603b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:39:00.083294 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f96a32-894a-4494-aba7-de90ce0603b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:39:00.083569 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f96a32-894a-4494-aba7-de90ce0603b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:39:00.083839 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f96a32-894a-4494-aba7-de90ce0603b5 admin admin] PATH_INFO: `/v3/users/34e4fe82fb76414dbd7ccfe8ff5b68b2` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:39:00.085053 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f96a32-894a-4494-aba7-de90ce0603b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34e4fe82fb76414dbd7ccfe8ff5b68b2)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:39:00.086950 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f96a32-894a-4494-aba7-de90ce0603b5 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:39:00.267727 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-a8f96a32-894a-4494-aba7-de90ce0603b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34e4fe82fb76414dbd7ccfe8ff5b68b2'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:39:00.274510 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-a8f96a32-894a-4494-aba7-de90ce0603b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34e4fe82fb76414dbd7ccfe8ff5b68b2'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:39:00.275173 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-a8f96a32-894a-4494-aba7-de90ce0603b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34e4fe82fb76414dbd7ccfe8ff5b68b2'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:39:00.275782 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-a8f96a32-894a-4494-aba7-de90ce0603b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34e4fe82fb76414dbd7ccfe8ff5b68b2'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:39:00.280910 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:39:00.289445 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-a8f96a32-894a-4494-aba7-de90ce0603b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34e4fe82fb76414dbd7ccfe8ff5b68b2'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:39:00.388703 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1421/2800] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:39:00 2022] DELETE /identity/v3/users/34e4fe82fb76414dbd7ccfe8ff5b68b2 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:39:00.389909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e3b04-90f1-445a-a9fe-56d26dddbb45 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:39:00.390192 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e3b04-90f1-445a-a9fe-56d26dddbb45 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:39:00.391069 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e3b04-90f1-445a-a9fe-56d26dddbb45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:39:00.410901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-8a4e3b04-90f1-445a-a9fe-56d26dddbb45 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:39:00.412102 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8a4e3b04-90f1-445a-a9fe-56d26dddbb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:39:00.463082 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:39:00.481367 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1380/2801] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:39:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:39:00.486207 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f96a32-894a-4494-aba7-de90ce0603b5 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:39:00.487666 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-a8f96a32-894a-4494-aba7-de90ce0603b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:39:00.548156 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f96a32-894a-4494-aba7-de90ce0603b5 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:39:00.551770 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f14322-3a09-4ee8-9d7e-f28d0f97cc7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:39:00.552391 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f14322-3a09-4ee8-9d7e-f28d0f97cc7f service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:39:00.552709 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f14322-3a09-4ee8-9d7e-f28d0f97cc7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:39:00.552983 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f14322-3a09-4ee8-9d7e-f28d0f97cc7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:39:00.561789 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f14322-3a09-4ee8-9d7e-f28d0f97cc7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:39:00.563666 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f14322-3a09-4ee8-9d7e-f28d0f97cc7f service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:39:00.573134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1422/2802] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:39:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:39:00.970280 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e3b04-90f1-445a-a9fe-56d26dddbb45 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:39:00.972066 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8a4e3b04-90f1-445a-a9fe-56d26dddbb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:39:00.988600 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e3b04-90f1-445a-a9fe-56d26dddbb45 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:39:00.993605 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d067ec-a98d-467e-bec7-cb6f32530ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:39:00.994311 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d067ec-a98d-467e-bec7-cb6f32530ff0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:39:00.994591 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d067ec-a98d-467e-bec7-cb6f32530ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:39:00.995000 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d067ec-a98d-467e-bec7-cb6f32530ff0 admin admin] PATH_INFO: `/v3/projects/c442c235b363456eb569a027799d949d` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:39:00.996236 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d067ec-a98d-467e-bec7-cb6f32530ff0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c442c235b363456eb569a027799d949d)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:39:00.998181 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d067ec-a98d-467e-bec7-cb6f32530ff0 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:39:01.112335 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c7d067ec-a98d-467e-bec7-cb6f32530ff0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c442c235b363456eb569a027799d949d', 'request_id': 'req-c7d067ec-a98d-467e-bec7-cb6f32530ff0'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:39:01.121820 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c7d067ec-a98d-467e-bec7-cb6f32530ff0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c442c235b363456eb569a027799d949d', 'request_id': 'req-c7d067ec-a98d-467e-bec7-cb6f32530ff0'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:39:01.189463 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c7d067ec-a98d-467e-bec7-cb6f32530ff0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c442c235b363456eb569a027799d949d', 'request_id': 'req-c7d067ec-a98d-467e-bec7-cb6f32530ff0'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:39:01.200351 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c7d067ec-a98d-467e-bec7-cb6f32530ff0 admin admin] The token cache is being invalidate because project c442c235b363456eb569a027799d949d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:39:01.200612 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c7d067ec-a98d-467e-bec7-cb6f32530ff0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:39:01.201217 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c7d067ec-a98d-467e-bec7-cb6f32530ff0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:39:01.202703 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1381/2803] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:39:00 2022] DELETE /identity/v3/projects/c442c235b363456eb569a027799d949d => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:39:10.568848 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=32,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:39:11.197174 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=30,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:39:15.961825 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d0ed1-24f8-4368-8a7c-87b60fcf4954 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:39:15.962318 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d0ed1-24f8-4368-8a7c-87b60fcf4954 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:39:15.965262 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d0ed1-24f8-4368-8a7c-87b60fcf4954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:39:15.981920 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-703d0ed1-24f8-4368-8a7c-87b60fcf4954 None None] MFA Rules not processed for user `f7b9ccd1b8cc4832a9c77513b5f21815`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:39:15.983107 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-703d0ed1-24f8-4368-8a7c-87b60fcf4954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:39:16.038425 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1423/2804] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:39:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:39:26.031643 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=25 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:39:31.756074 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d067ec-a98d-467e-bec7-cb6f32530ff0 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:39:31.757504 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c7d067ec-a98d-467e-bec7-cb6f32530ff0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:39:31.825429 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d067ec-a98d-467e-bec7-cb6f32530ff0 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:39:31.830392 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b981cab7-7a96-4015-8d62-e929439e71ee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff761f6f81a46098f94bd2ec2be9c1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:39:31.831006 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981cab7-7a96-4015-8d62-e929439e71ee service placement] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:39:31.831283 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981cab7-7a96-4015-8d62-e929439e71ee service placement] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:39:31.831555 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981cab7-7a96-4015-8d62-e929439e71ee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:39:31.833292 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-b981cab7-7a96-4015-8d62-e929439e71ee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:39:31.845103 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b981cab7-7a96-4015-8d62-e929439e71ee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:39:31.846719 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b981cab7-7a96-4015-8d62-e929439e71ee service placement] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:39:31.856558 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1382/2805] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:39:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:39:35.969116 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d0ed1-24f8-4368-8a7c-87b60fcf4954 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:39:35.971078 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-703d0ed1-24f8-4368-8a7c-87b60fcf4954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:39:36.026440 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d0ed1-24f8-4368-8a7c-87b60fcf4954 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:39:36.030018 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c506fb9-1980-411a-b329-d890579ea5b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:39:36.030574 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c506fb9-1980-411a-b329-d890579ea5b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:39:36.030850 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c506fb9-1980-411a-b329-d890579ea5b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:39:36.031118 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c506fb9-1980-411a-b329-d890579ea5b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:39:36.033414 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-4c506fb9-1980-411a-b329-d890579ea5b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:39:36.095983 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c506fb9-1980-411a-b329-d890579ea5b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:39:36.097424 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c506fb9-1980-411a-b329-d890579ea5b4 service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:39:36.106740 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1424/2806] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:39:35 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:39:41.852475 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=32 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:39:46.102610 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=50 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:40:16.877140 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a82ce53-2913-43d5-bdbe-e5858b26661c None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:40:16.877682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a82ce53-2913-43d5-bdbe-e5858b26661c None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:40:16.877682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a82ce53-2913-43d5-bdbe-e5858b26661c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:40:16.897873 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-6a82ce53-2913-43d5-bdbe-e5858b26661c None None] MFA Rules not processed for user `0cb8261b328c4cd8956762971fefceee`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:40:16.899075 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-6a82ce53-2913-43d5-bdbe-e5858b26661c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:40:16.952834 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1383/2807] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:40:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:40:17.657678 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c506fb9-1980-411a-b329-d890579ea5b4 service neutron] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:40:17.666904 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c506fb9-1980-411a-b329-d890579ea5b4 service neutron] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:40:17.670572 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac0c826-fc8f-46a1-9926-288b09ef8c71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:40:17.671175 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0c826-fc8f-46a1-9926-288b09ef8c71 service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:40:17.671456 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0c826-fc8f-46a1-9926-288b09ef8c71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:40:17.671727 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0c826-fc8f-46a1-9926-288b09ef8c71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:40:17.673529 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-aac0c826-fc8f-46a1-9926-288b09ef8c71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:40:17.747796 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac0c826-fc8f-46a1-9926-288b09ef8c71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:40:17.749277 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac0c826-fc8f-46a1-9926-288b09ef8c71 service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:40:17.761179 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1425/2808] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:40:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:40:18.869935 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a82ce53-2913-43d5-bdbe-e5858b26661c None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:40:18.880664 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a82ce53-2913-43d5-bdbe-e5858b26661c None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:40:18.884306 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17360db4-e3f1-45f6-a1a5-966c0dcba1f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:40:18.885037 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17360db4-e3f1-45f6-a1a5-966c0dcba1f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:40:18.885312 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17360db4-e3f1-45f6-a1a5-966c0dcba1f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:40:18.885580 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17360db4-e3f1-45f6-a1a5-966c0dcba1f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:40:18.887333 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-17360db4-e3f1-45f6-a1a5-966c0dcba1f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:40:18.901792 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17360db4-e3f1-45f6-a1a5-966c0dcba1f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:40:18.903281 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17360db4-e3f1-45f6-a1a5-966c0dcba1f6 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:40:18.914508 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1384/2809] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:40:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:40:27.756502 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=34 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:40:28.909317 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=31 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:41:31.361354 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac0c826-fc8f-46a1-9926-288b09ef8c71 service neutron] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:41:31.362833 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-aac0c826-fc8f-46a1-9926-288b09ef8c71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:41:31.373413 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac0c826-fc8f-46a1-9926-288b09ef8c71 service neutron] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:41:31.376973 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e5854-93de-4763-bc26-ba112218e00b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:41:31.377572 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e5854-93de-4763-bc26-ba112218e00b service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:41:31.377845 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e5854-93de-4763-bc26-ba112218e00b service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:41:31.378111 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e5854-93de-4763-bc26-ba112218e00b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:41:31.379767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-628e5854-93de-4763-bc26-ba112218e00b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:41:31.389453 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e5854-93de-4763-bc26-ba112218e00b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:41:31.390856 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e5854-93de-4763-bc26-ba112218e00b service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:41:31.398818 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1426/2810] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:41:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:41:41.395084 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=6 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:41:47.008930 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17360db4-e3f1-45f6-a1a5-966c0dcba1f6 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:41:47.010403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-17360db4-e3f1-45f6-a1a5-966c0dcba1f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:41:47.025081 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17360db4-e3f1-45f6-a1a5-966c0dcba1f6 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:41:47.031592 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1ae03-e787-476c-ae89-026b1288d7b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:41:47.032138 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1ae03-e787-476c-ae89-026b1288d7b4 service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:41:47.032446 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1ae03-e787-476c-ae89-026b1288d7b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:41:47.032741 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1ae03-e787-476c-ae89-026b1288d7b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:41:47.034471 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8cc1ae03-e787-476c-ae89-026b1288d7b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:41:47.044776 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc1ae03-e787-476c-ae89-026b1288d7b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:41:47.046337 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc1ae03-e787-476c-ae89-026b1288d7b4 service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:41:47.055197 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1385/2811] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:41:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 14:41:55.444710 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e5854-93de-4763-bc26-ba112218e00b service neutron] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:41:55.458313 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e5854-93de-4763-bc26-ba112218e00b service neutron] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:41:55.463769 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc931819-d739-45aa-91ff-632e425e869a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:41:55.464378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc931819-d739-45aa-91ff-632e425e869a service nova] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:41:55.464681 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc931819-d739-45aa-91ff-632e425e869a service nova] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:41:55.464953 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc931819-d739-45aa-91ff-632e425e869a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:41:55.468293 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-bc931819-d739-45aa-91ff-632e425e869a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:41:55.480741 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc931819-d739-45aa-91ff-632e425e869a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:41:55.486863 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc931819-d739-45aa-91ff-632e425e869a service nova] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:41:55.499742 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1427/2812] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:41:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:41:57.051195 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=6 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:41:57.384977 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d0399-3188-4850-bc24-509e77e1d651 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:41:57.385260 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d0399-3188-4850-bc24-509e77e1d651 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:41:57.385534 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d0399-3188-4850-bc24-509e77e1d651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:41:57.404376 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-738d0399-3188-4850-bc24-509e77e1d651 None None] MFA Rules not processed for user `f7b9ccd1b8cc4832a9c77513b5f21815`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:41:57.405600 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-738d0399-3188-4850-bc24-509e77e1d651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:41:57.412375 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1386/2813] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:41:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:41:59.410265 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc931819-d739-45aa-91ff-632e425e869a service nova] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:41:59.422468 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc931819-d739-45aa-91ff-632e425e869a service nova] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:41:59.427889 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d1177-e413-415c-bfac-cd99f3d73236 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff761f6f81a46098f94bd2ec2be9c1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:41:59.428526 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d1177-e413-415c-bfac-cd99f3d73236 service placement] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:41:59.428803 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d1177-e413-415c-bfac-cd99f3d73236 service placement] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:41:59.429069 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d1177-e413-415c-bfac-cd99f3d73236 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:41:59.430927 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-315d1177-e413-415c-bfac-cd99f3d73236 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:41:59.442957 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315d1177-e413-415c-bfac-cd99f3d73236 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:41:59.444545 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315d1177-e413-415c-bfac-cd99f3d73236 service placement] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:41:59.454504 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1428/2814] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:41:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:42:07.399714 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:42:09.176521 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e21fa-9ff3-4e9b-a061-242d3f4b4928 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:42:09.177228 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e21fa-9ff3-4e9b-a061-242d3f4b4928 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:42:09.177228 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e21fa-9ff3-4e9b-a061-242d3f4b4928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:42:09.196197 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-c57e21fa-9ff3-4e9b-a061-242d3f4b4928 None None] MFA Rules not processed for user `0cb8261b328c4cd8956762971fefceee`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:42:09.197426 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c57e21fa-9ff3-4e9b-a061-242d3f4b4928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:42:09.204966 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1387/2815] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:42:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:42:09.450134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=12 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:42:19.191637 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:42:24.428940 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d1177-e413-415c-bfac-cd99f3d73236 service placement] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:42:24.441666 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d1177-e413-415c-bfac-cd99f3d73236 service placement] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:42:24.447417 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f98c063-b088-4086-b72c-423a5f654851 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:42:24.448029 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98c063-b088-4086-b72c-423a5f654851 service nova] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:42:24.448305 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98c063-b088-4086-b72c-423a5f654851 service nova] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:42:24.448618 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98c063-b088-4086-b72c-423a5f654851 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:42:24.450406 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-7f98c063-b088-4086-b72c-423a5f654851 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:42:24.463969 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f98c063-b088-4086-b72c-423a5f654851 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:42:24.465802 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f98c063-b088-4086-b72c-423a5f654851 service nova] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:42:24.478418 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1429/2816] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:42:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:42:34.473341 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=6 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:44:03.184101 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e21fa-9ff3-4e9b-a061-242d3f4b4928 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:03.221182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e21fa-9ff3-4e9b-a061-242d3f4b4928 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:03.224785 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae855d38-d0d1-45d6-aaad-f6650816b42f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:03.225391 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae855d38-d0d1-45d6-aaad-f6650816b42f service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:03.225694 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae855d38-d0d1-45d6-aaad-f6650816b42f service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:03.225964 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae855d38-d0d1-45d6-aaad-f6650816b42f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:03.227702 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ae855d38-d0d1-45d6-aaad-f6650816b42f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:03.337620 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae855d38-d0d1-45d6-aaad-f6650816b42f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:03.339295 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae855d38-d0d1-45d6-aaad-f6650816b42f service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:03.359160 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1388/2817] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:44:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:44:06.720481 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f98c063-b088-4086-b72c-423a5f654851 service nova] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:06.764379 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f98c063-b088-4086-b72c-423a5f654851 service nova] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:06.769673 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2bc1b0-e2e1-4ddf-bde7-5a428de745f3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff761f6f81a46098f94bd2ec2be9c1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:06.770287 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2bc1b0-e2e1-4ddf-bde7-5a428de745f3 service placement] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:06.770568 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2bc1b0-e2e1-4ddf-bde7-5a428de745f3 service placement] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:06.770842 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2bc1b0-e2e1-4ddf-bde7-5a428de745f3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:06.772648 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-bc2bc1b0-e2e1-4ddf-bde7-5a428de745f3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:06.784797 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2bc1b0-e2e1-4ddf-bde7-5a428de745f3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:06.786351 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2bc1b0-e2e1-4ddf-bde7-5a428de745f3 service placement] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:06.797076 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1430/2818] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:44:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:44:06.836902 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae855d38-d0d1-45d6-aaad-f6650816b42f service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:06.838693 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ae855d38-d0d1-45d6-aaad-f6650816b42f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:06.851824 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae855d38-d0d1-45d6-aaad-f6650816b42f service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:06.856600 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dc44f5-24b5-429a-82e7-1b5cb9829006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:06.857248 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc44f5-24b5-429a-82e7-1b5cb9829006 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:06.857530 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc44f5-24b5-429a-82e7-1b5cb9829006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:06.858349 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc44f5-24b5-429a-82e7-1b5cb9829006 admin admin] PATH_INFO: `/v3/users/0cb8261b328c4cd8956762971fefceee` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:06.859682 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dc44f5-24b5-429a-82e7-1b5cb9829006 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cb8261b328c4cd8956762971fefceee)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:06.861691 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dc44f5-24b5-429a-82e7-1b5cb9829006 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:07.162638 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-08dc44f5-24b5-429a-82e7-1b5cb9829006 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cb8261b328c4cd8956762971fefceee'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:07.175178 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-08dc44f5-24b5-429a-82e7-1b5cb9829006 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cb8261b328c4cd8956762971fefceee'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:07.176974 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:44:07.268985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-08dc44f5-24b5-429a-82e7-1b5cb9829006 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cb8261b328c4cd8956762971fefceee'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:07.269610 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-08dc44f5-24b5-429a-82e7-1b5cb9829006 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cb8261b328c4cd8956762971fefceee'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:07.276817 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-08dc44f5-24b5-429a-82e7-1b5cb9829006 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cb8261b328c4cd8956762971fefceee'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:07.283850 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1389/2819] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:44:06 2022] DELETE /identity/v3/users/0cb8261b328c4cd8956762971fefceee => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:44:07.551136 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2bc1b0-e2e1-4ddf-bde7-5a428de745f3 service placement] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:07.563411 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2bc1b0-e2e1-4ddf-bde7-5a428de745f3 service placement] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:07.622525 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d640734-1cca-4872-acfd-6d713fe396b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:07.623152 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d640734-1cca-4872-acfd-6d713fe396b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:07.623436 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d640734-1cca-4872-acfd-6d713fe396b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:07.623910 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d640734-1cca-4872-acfd-6d713fe396b2 admin admin] PATH_INFO: `/v3/projects/bc63bf4a15624098aa5568cad2c003ae` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:07.625158 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d640734-1cca-4872-acfd-6d713fe396b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc63bf4a15624098aa5568cad2c003ae)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:07.627035 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d640734-1cca-4872-acfd-6d713fe396b2 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:07.720147 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-6d640734-1cca-4872-acfd-6d713fe396b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc63bf4a15624098aa5568cad2c003ae', 'request_id': 'req-6d640734-1cca-4872-acfd-6d713fe396b2'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:07.726162 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-6d640734-1cca-4872-acfd-6d713fe396b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc63bf4a15624098aa5568cad2c003ae', 'request_id': 'req-6d640734-1cca-4872-acfd-6d713fe396b2'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:07.731640 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-6d640734-1cca-4872-acfd-6d713fe396b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc63bf4a15624098aa5568cad2c003ae', 'request_id': 'req-6d640734-1cca-4872-acfd-6d713fe396b2'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:07.782187 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-6d640734-1cca-4872-acfd-6d713fe396b2 admin admin] The token cache is being invalidate because project bc63bf4a15624098aa5568cad2c003ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:44:07.782496 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-6d640734-1cca-4872-acfd-6d713fe396b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:07.783106 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-6d640734-1cca-4872-acfd-6d713fe396b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:07.784632 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1431/2820] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:44:07 2022] DELETE /identity/v3/projects/bc63bf4a15624098aa5568cad2c003ae => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:44:07.789647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae1342-5c01-4c51-ad55-189c07f1f8da None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:07.790688 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae1342-5c01-4c51-ad55-189c07f1f8da None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:07.790688 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae1342-5c01-4c51-ad55-189c07f1f8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:07.850333 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-48ae1342-5c01-4c51-ad55-189c07f1f8da None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:44:07.851546 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-48ae1342-5c01-4c51-ad55-189c07f1f8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:07.910218 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1390/2821] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:44:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:44:07.919375 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b00ce-b50c-4cbd-8689-6d08335d3d2b None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:07.919661 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b00ce-b50c-4cbd-8689-6d08335d3d2b None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:07.919936 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b00ce-b50c-4cbd-8689-6d08335d3d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:07.940843 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-862b00ce-b50c-4cbd-8689-6d08335d3d2b None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:44:07.942054 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-862b00ce-b50c-4cbd-8689-6d08335d3d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:07.949918 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1432/2822] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:44:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:44:07.954320 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ae1342-5c01-4c51-ad55-189c07f1f8da None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:07.964132 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ae1342-5c01-4c51-ad55-189c07f1f8da None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:07.969280 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93117bf-05ad-4acf-be92-bde92271f2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:07.969875 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93117bf-05ad-4acf-be92-bde92271f2d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:07.970204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93117bf-05ad-4acf-be92-bde92271f2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:07.970420 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93117bf-05ad-4acf-be92-bde92271f2d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:07.971103 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93117bf-05ad-4acf-be92-bde92271f2d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:44:07.971405 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93117bf-05ad-4acf-be92-bde92271f2d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:07.973036 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93117bf-05ad-4acf-be92-bde92271f2d0 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:07.984975 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1391/2823] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:44:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:44:07.989186 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b00ce-b50c-4cbd-8689-6d08335d3d2b None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:07.998315 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b00ce-b50c-4cbd-8689-6d08335d3d2b None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:08.003183 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6f0be-5a1a-4ae9-b7f3-1d9846ca0317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:08.003780 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6f0be-5a1a-4ae9-b7f3-1d9846ca0317 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:08.004058 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6f0be-5a1a-4ae9-b7f3-1d9846ca0317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:08.004328 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6f0be-5a1a-4ae9-b7f3-1d9846ca0317 admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:08.005046 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd6f0be-5a1a-4ae9-b7f3-1d9846ca0317 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:08.006937 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd6f0be-5a1a-4ae9-b7f3-1d9846ca0317 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:08.116941 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1433/2824] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:44:07 2022] POST /identity/v3/projects => generated 379 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:44:08.121109 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93117bf-05ad-4acf-be92-bde92271f2d0 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:08.131191 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93117bf-05ad-4acf-be92-bde92271f2d0 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:08.186310 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94fd887-6007-4024-9604-786f956a1d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:08.186940 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94fd887-6007-4024-9604-786f956a1d04 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:08.187215 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94fd887-6007-4024-9604-786f956a1d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:08.187506 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94fd887-6007-4024-9604-786f956a1d04 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:08.188137 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94fd887-6007-4024-9604-786f956a1d04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:08.190053 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94fd887-6007-4024-9604-786f956a1d04 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:08.274778 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1392/2825] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:44:08 2022] POST /identity/v3/users => generated 290 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:44:08.278834 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6f0be-5a1a-4ae9-b7f3-1d9846ca0317 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:08.290031 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6f0be-5a1a-4ae9-b7f3-1d9846ca0317 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:08.294667 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ca654-c653-4436-bf29-58193db10705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:08.295265 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ca654-c653-4436-bf29-58193db10705 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:08.296480 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ca654-c653-4436-bf29-58193db10705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:08.296851 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ca654-c653-4436-bf29-58193db10705 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:08.297536 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815ca654-c653-4436-bf29-58193db10705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:08.299099 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815ca654-c653-4436-bf29-58193db10705 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:08.308822 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1434/2826] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:44:08.315164 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94fd887-6007-4024-9604-786f956a1d04 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:08.325036 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94fd887-6007-4024-9604-786f956a1d04 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:08.329553 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4bc71-e982-43ad-b0d8-3eaa94b415af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:08.330194 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4bc71-e982-43ad-b0d8-3eaa94b415af admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:08.330473 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4bc71-e982-43ad-b0d8-3eaa94b415af admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:08.330745 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4bc71-e982-43ad-b0d8-3eaa94b415af admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:08.331429 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd4bc71-e982-43ad-b0d8-3eaa94b415af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:08.333088 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd4bc71-e982-43ad-b0d8-3eaa94b415af admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:08.344327 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1393/2827] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:44:08.348361 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ca654-c653-4436-bf29-58193db10705 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:08.358967 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ca654-c653-4436-bf29-58193db10705 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:08.363577 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e0a48c-57bc-421a-b5be-95fc55c6769b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:08.364221 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e0a48c-57bc-421a-b5be-95fc55c6769b admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:08.364524 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e0a48c-57bc-421a-b5be-95fc55c6769b admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:08.364808 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e0a48c-57bc-421a-b5be-95fc55c6769b admin admin] PATH_INFO: `/v3/projects/9047af13b6ee40a4a30559fcb91f82cb/users/6db0dba1a9dd4388b3d9c879761c360b/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:08.366955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:44:08.379557 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e0a48c-57bc-421a-b5be-95fc55c6769b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9047af13b6ee40a4a30559fcb91f82cb, user_id=6db0dba1a9dd4388b3d9c879761c360b, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:08.382218 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e0a48c-57bc-421a-b5be-95fc55c6769b admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:08.469724 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1435/2828] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:44:08 2022] PUT /identity/v3/projects/9047af13b6ee40a4a30559fcb91f82cb/users/6db0dba1a9dd4388b3d9c879761c360b/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:44:08.480068 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f709c16-ff2d-4f2e-90c6-79ac9162c318 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:08.480350 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f709c16-ff2d-4f2e-90c6-79ac9162c318 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:08.480654 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f709c16-ff2d-4f2e-90c6-79ac9162c318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:08.508099 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-0f709c16-ff2d-4f2e-90c6-79ac9162c318 None None] MFA Rules not processed for user `6db0dba1a9dd4388b3d9c879761c360b`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:44:08.509329 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-0f709c16-ff2d-4f2e-90c6-79ac9162c318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:08.515418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:44:08.576377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1394/2829] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:44:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:44:08.581642 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e0a48c-57bc-421a-b5be-95fc55c6769b admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:08.642182 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e0a48c-57bc-421a-b5be-95fc55c6769b admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:08.646713 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf9418-5916-42ca-bc72-f81f8c62368c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:08.647319 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf9418-5916-42ca-bc72-f81f8c62368c admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:08.647591 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf9418-5916-42ca-bc72-f81f8c62368c admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:08.647856 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf9418-5916-42ca-bc72-f81f8c62368c admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:08.648536 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf9418-5916-42ca-bc72-f81f8c62368c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:08.650407 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf9418-5916-42ca-bc72-f81f8c62368c admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:08.750058 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1436/2830] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:44:08 2022] POST /identity/v3/projects => generated 381 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:44:08.754209 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f709c16-ff2d-4f2e-90c6-79ac9162c318 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:08.766434 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f709c16-ff2d-4f2e-90c6-79ac9162c318 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:08.823149 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d5644-846d-4bf0-8e4b-3209316319cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:08.823683 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d5644-846d-4bf0-8e4b-3209316319cc admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:08.823958 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d5644-846d-4bf0-8e4b-3209316319cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:08.824226 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d5644-846d-4bf0-8e4b-3209316319cc admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:08.824912 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d5644-846d-4bf0-8e4b-3209316319cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:08.826819 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d5644-846d-4bf0-8e4b-3209316319cc admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:08.927157 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1395/2831] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:44:08 2022] POST /identity/v3/users => generated 297 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:44:08.931135 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf9418-5916-42ca-bc72-f81f8c62368c admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:08.941685 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf9418-5916-42ca-bc72-f81f8c62368c admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:08.946305 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae11df-bdb1-4a49-acc1-1c917d2e79f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:08.946903 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae11df-bdb1-4a49-acc1-1c917d2e79f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:08.953257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae11df-bdb1-4a49-acc1-1c917d2e79f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:08.953257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae11df-bdb1-4a49-acc1-1c917d2e79f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:08.953257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ae11df-bdb1-4a49-acc1-1c917d2e79f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:08.953257 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ae11df-bdb1-4a49-acc1-1c917d2e79f7 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:08.960529 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1437/2832] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:44:08.965033 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d5644-846d-4bf0-8e4b-3209316319cc admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:08.975508 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d5644-846d-4bf0-8e4b-3209316319cc admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:08.980001 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547b0a8-9eb1-4599-9c2b-a021e839246b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:08.980674 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547b0a8-9eb1-4599-9c2b-a021e839246b admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:08.980956 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547b0a8-9eb1-4599-9c2b-a021e839246b admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:08.981225 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547b0a8-9eb1-4599-9c2b-a021e839246b admin admin] PATH_INFO: `/v3/projects/9d6fdd974f5a40c2bce2a629b8e923ab/users/75c59abf25bf400391b75282ff8c07f2/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:08.996660 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5547b0a8-9eb1-4599-9c2b-a021e839246b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d6fdd974f5a40c2bce2a629b8e923ab, user_id=75c59abf25bf400391b75282ff8c07f2, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:08.999480 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5547b0a8-9eb1-4599-9c2b-a021e839246b admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:09.069033 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1396/2833] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:44:08 2022] PUT /identity/v3/projects/9d6fdd974f5a40c2bce2a629b8e923ab/users/75c59abf25bf400391b75282ff8c07f2/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:44:09.072955 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae11df-bdb1-4a49-acc1-1c917d2e79f7 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:09.083393 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae11df-bdb1-4a49-acc1-1c917d2e79f7 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:09.139757 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56eacfb-0fe3-415e-abee-39bfacc21ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:09.140578 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56eacfb-0fe3-415e-abee-39bfacc21ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:09.140856 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56eacfb-0fe3-415e-abee-39bfacc21ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:09.141127 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56eacfb-0fe3-415e-abee-39bfacc21ab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:09.141803 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56eacfb-0fe3-415e-abee-39bfacc21ab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:09.143369 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56eacfb-0fe3-415e-abee-39bfacc21ab0 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:09.152927 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1438/2834] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:44:09.157721 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547b0a8-9eb1-4599-9c2b-a021e839246b admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:09.167893 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547b0a8-9eb1-4599-9c2b-a021e839246b admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:09.172576 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e687b64-c7c2-417d-aeb6-53cca8a41b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:09.173226 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e687b64-c7c2-417d-aeb6-53cca8a41b9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:09.173503 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e687b64-c7c2-417d-aeb6-53cca8a41b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:09.173775 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e687b64-c7c2-417d-aeb6-53cca8a41b9b admin admin] PATH_INFO: `/v3/projects/9d6fdd974f5a40c2bce2a629b8e923ab/users/75c59abf25bf400391b75282ff8c07f2/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:09.175721 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e687b64-c7c2-417d-aeb6-53cca8a41b9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d6fdd974f5a40c2bce2a629b8e923ab, user_id=75c59abf25bf400391b75282ff8c07f2, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:09.178381 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e687b64-c7c2-417d-aeb6-53cca8a41b9b admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:09.188874 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1397/2835] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:44:09 2022] PUT /identity/v3/projects/9d6fdd974f5a40c2bce2a629b8e923ab/users/75c59abf25bf400391b75282ff8c07f2/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:44:09.198991 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b814d-7772-48ac-8bdc-931376a5d8b5 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:09.199274 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b814d-7772-48ac-8bdc-931376a5d8b5 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:09.199546 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b814d-7772-48ac-8bdc-931376a5d8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:09.228176 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-af5b814d-7772-48ac-8bdc-931376a5d8b5 None None] MFA Rules not processed for user `75c59abf25bf400391b75282ff8c07f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:44:09.229399 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-af5b814d-7772-48ac-8bdc-931376a5d8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:09.271813 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:44:09.297468 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1439/2836] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:44:09 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:44:09.308607 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e687b64-c7c2-417d-aeb6-53cca8a41b9b admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:09.310323 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-7e687b64-c7c2-417d-aeb6-53cca8a41b9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:09.376433 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:44:09.388972 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e687b64-c7c2-417d-aeb6-53cca8a41b9b admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:09.393394 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0313cd53-9953-4eac-8dea-b9e6cc83ec77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:09.394000 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313cd53-9953-4eac-8dea-b9e6cc83ec77 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:09.394272 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313cd53-9953-4eac-8dea-b9e6cc83ec77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:09.394541 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313cd53-9953-4eac-8dea-b9e6cc83ec77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:09.404080 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0313cd53-9953-4eac-8dea-b9e6cc83ec77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:09.406198 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0313cd53-9953-4eac-8dea-b9e6cc83ec77 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:09.417185 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1398/2837] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:44:09 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:44:09.637408 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b814d-7772-48ac-8bdc-931376a5d8b5 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:09.647992 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b814d-7772-48ac-8bdc-931376a5d8b5 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:09.651505 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f864e01-0459-49fc-a6df-5cba73d90085 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:09.652103 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f864e01-0459-49fc-a6df-5cba73d90085 service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:09.652378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f864e01-0459-49fc-a6df-5cba73d90085 service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:09.652675 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f864e01-0459-49fc-a6df-5cba73d90085 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:09.710576 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f864e01-0459-49fc-a6df-5cba73d90085 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:09.712016 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f864e01-0459-49fc-a6df-5cba73d90085 service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:09.721549 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1440/2838] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:44:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:44:19.429175 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=8 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:44:19.717111 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=38 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:44:22.008597 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0313cd53-9953-4eac-8dea-b9e6cc83ec77 service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:22.069033 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0313cd53-9953-4eac-8dea-b9e6cc83ec77 service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:22.073700 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293d906-b5e6-4678-a6eb-02741efe7469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:22.074335 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293d906-b5e6-4678-a6eb-02741efe7469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:22.074614 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293d906-b5e6-4678-a6eb-02741efe7469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:22.074887 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293d906-b5e6-4678-a6eb-02741efe7469 admin admin] PATH_INFO: `/v3/users/6db0dba1a9dd4388b3d9c879761c360b` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:22.076052 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293d906-b5e6-4678-a6eb-02741efe7469 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6db0dba1a9dd4388b3d9c879761c360b)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:22.078002 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293d906-b5e6-4678-a6eb-02741efe7469 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:22.259161 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-3293d906-b5e6-4678-a6eb-02741efe7469 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6db0dba1a9dd4388b3d9c879761c360b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:22.271861 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-3293d906-b5e6-4678-a6eb-02741efe7469 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6db0dba1a9dd4388b3d9c879761c360b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:22.337067 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-3293d906-b5e6-4678-a6eb-02741efe7469 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6db0dba1a9dd4388b3d9c879761c360b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:22.337894 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-3293d906-b5e6-4678-a6eb-02741efe7469 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6db0dba1a9dd4388b3d9c879761c360b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:22.346174 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-3293d906-b5e6-4678-a6eb-02741efe7469 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6db0dba1a9dd4388b3d9c879761c360b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:22.352127 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1399/2839] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:44:22 2022] DELETE /identity/v3/users/6db0dba1a9dd4388b3d9c879761c360b => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:44:22.356813 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a85a2-be1d-4037-a8e9-0f8ddb106677 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:22.357387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a85a2-be1d-4037-a8e9-0f8ddb106677 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:22.357387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a85a2-be1d-4037-a8e9-0f8ddb106677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:22.377029 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-cf6a85a2-be1d-4037-a8e9-0f8ddb106677 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:44:22.382148 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-cf6a85a2-be1d-4037-a8e9-0f8ddb106677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:22.435592 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1441/2840] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:44:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:44:22.445237 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293d906-b5e6-4678-a6eb-02741efe7469 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:22.446638 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-3293d906-b5e6-4678-a6eb-02741efe7469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:22.516051 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293d906-b5e6-4678-a6eb-02741efe7469 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:22.520067 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd1431-8dba-40fa-a6f6-bf3180ca13d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:22.520720 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd1431-8dba-40fa-a6f6-bf3180ca13d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:22.521006 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd1431-8dba-40fa-a6f6-bf3180ca13d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:22.521277 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd1431-8dba-40fa-a6f6-bf3180ca13d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:22.530941 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cd1431-8dba-40fa-a6f6-bf3180ca13d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:22.533030 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cd1431-8dba-40fa-a6f6-bf3180ca13d7 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:22.542370 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1400/2841] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:44:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:44:22.956899 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a85a2-be1d-4037-a8e9-0f8ddb106677 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:22.961225 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-cf6a85a2-be1d-4037-a8e9-0f8ddb106677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:22.971556 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a85a2-be1d-4037-a8e9-0f8ddb106677 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:22.976860 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a77cd9-9d85-461b-96c2-4dc77dbab426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:22.977509 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a77cd9-9d85-461b-96c2-4dc77dbab426 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:22.977788 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a77cd9-9d85-461b-96c2-4dc77dbab426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:22.978058 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a77cd9-9d85-461b-96c2-4dc77dbab426 admin admin] PATH_INFO: `/v3/projects/9047af13b6ee40a4a30559fcb91f82cb` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:22.979242 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a77cd9-9d85-461b-96c2-4dc77dbab426 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9047af13b6ee40a4a30559fcb91f82cb)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:22.981173 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a77cd9-9d85-461b-96c2-4dc77dbab426 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:23.091928 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-f9a77cd9-9d85-461b-96c2-4dc77dbab426 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9047af13b6ee40a4a30559fcb91f82cb', 'request_id': 'req-f9a77cd9-9d85-461b-96c2-4dc77dbab426'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:23.098066 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-f9a77cd9-9d85-461b-96c2-4dc77dbab426 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9047af13b6ee40a4a30559fcb91f82cb', 'request_id': 'req-f9a77cd9-9d85-461b-96c2-4dc77dbab426'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:23.103679 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-f9a77cd9-9d85-461b-96c2-4dc77dbab426 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9047af13b6ee40a4a30559fcb91f82cb', 'request_id': 'req-f9a77cd9-9d85-461b-96c2-4dc77dbab426'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:23.170599 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-f9a77cd9-9d85-461b-96c2-4dc77dbab426 admin admin] The token cache is being invalidate because project 9047af13b6ee40a4a30559fcb91f82cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:44:23.170890 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-f9a77cd9-9d85-461b-96c2-4dc77dbab426 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:23.171505 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-f9a77cd9-9d85-461b-96c2-4dc77dbab426 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:23.173082 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1442/2842] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:44:22 2022] DELETE /identity/v3/projects/9047af13b6ee40a4a30559fcb91f82cb => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:44:23.177075 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd1431-8dba-40fa-a6f6-bf3180ca13d7 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:23.186673 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd1431-8dba-40fa-a6f6-bf3180ca13d7 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:23.195336 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:44:23.243696 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3839f8-9df1-4eef-8fd0-7ce37a34ecd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:23.244264 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3839f8-9df1-4eef-8fd0-7ce37a34ecd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:23.244568 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3839f8-9df1-4eef-8fd0-7ce37a34ecd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:23.244838 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3839f8-9df1-4eef-8fd0-7ce37a34ecd2 admin admin] PATH_INFO: `/v3/users/75c59abf25bf400391b75282ff8c07f2` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:23.245990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3839f8-9df1-4eef-8fd0-7ce37a34ecd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75c59abf25bf400391b75282ff8c07f2)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:23.247893 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3839f8-9df1-4eef-8fd0-7ce37a34ecd2 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:23.424956 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-6b3839f8-9df1-4eef-8fd0-7ce37a34ecd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75c59abf25bf400391b75282ff8c07f2'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:23.436575 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-6b3839f8-9df1-4eef-8fd0-7ce37a34ecd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75c59abf25bf400391b75282ff8c07f2'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:23.508909 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-6b3839f8-9df1-4eef-8fd0-7ce37a34ecd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75c59abf25bf400391b75282ff8c07f2'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:23.509538 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-6b3839f8-9df1-4eef-8fd0-7ce37a34ecd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75c59abf25bf400391b75282ff8c07f2'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:23.517920 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-6b3839f8-9df1-4eef-8fd0-7ce37a34ecd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75c59abf25bf400391b75282ff8c07f2'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:23.529440 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1401/2843] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:44:23 2022] DELETE /identity/v3/users/75c59abf25bf400391b75282ff8c07f2 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:44:23.812400 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a77cd9-9d85-461b-96c2-4dc77dbab426 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:23.814478 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-f9a77cd9-9d85-461b-96c2-4dc77dbab426 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:23.873492 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a77cd9-9d85-461b-96c2-4dc77dbab426 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:23.910125 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:44:23.920532 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef7196-3cec-44cc-be57-eaf702474ce0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:23.921143 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef7196-3cec-44cc-be57-eaf702474ce0 service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:23.921418 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef7196-3cec-44cc-be57-eaf702474ce0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:23.921684 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef7196-3cec-44cc-be57-eaf702474ce0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:23.924234 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-b6ef7196-3cec-44cc-be57-eaf702474ce0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:23.944063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3839f8-9df1-4eef-8fd0-7ce37a34ecd2 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:23.944063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-6b3839f8-9df1-4eef-8fd0-7ce37a34ecd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:23.987545 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef7196-3cec-44cc-be57-eaf702474ce0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:23.989027 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef7196-3cec-44cc-be57-eaf702474ce0 service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:23.998669 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1443/2844] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:44:23 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:44:24.027053 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3839f8-9df1-4eef-8fd0-7ce37a34ecd2 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:24.034024 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524761df-dac4-413b-8a72-dd697524c4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:24.034024 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524761df-dac4-413b-8a72-dd697524c4bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:24.034024 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524761df-dac4-413b-8a72-dd697524c4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:24.034024 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524761df-dac4-413b-8a72-dd697524c4bc admin admin] PATH_INFO: `/v3/projects/9d6fdd974f5a40c2bce2a629b8e923ab` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:24.035107 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524761df-dac4-413b-8a72-dd697524c4bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d6fdd974f5a40c2bce2a629b8e923ab)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:24.037120 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524761df-dac4-413b-8a72-dd697524c4bc admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:24.131745 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:44:24.145241 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-524761df-dac4-413b-8a72-dd697524c4bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d6fdd974f5a40c2bce2a629b8e923ab', 'request_id': 'req-524761df-dac4-413b-8a72-dd697524c4bc'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:24.151241 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-524761df-dac4-413b-8a72-dd697524c4bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d6fdd974f5a40c2bce2a629b8e923ab', 'request_id': 'req-524761df-dac4-413b-8a72-dd697524c4bc'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:24.199990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-524761df-dac4-413b-8a72-dd697524c4bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d6fdd974f5a40c2bce2a629b8e923ab', 'request_id': 'req-524761df-dac4-413b-8a72-dd697524c4bc'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:24.211181 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-524761df-dac4-413b-8a72-dd697524c4bc admin admin] The token cache is being invalidate because project 9d6fdd974f5a40c2bce2a629b8e923ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:44:24.211842 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-524761df-dac4-413b-8a72-dd697524c4bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:24.212130 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-524761df-dac4-413b-8a72-dd697524c4bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:24.213929 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1402/2845] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:44:23 2022] DELETE /identity/v3/projects/9d6fdd974f5a40c2bce2a629b8e923ab => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:44:24.220593 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a967d-789e-4b00-8cc2-bfdc86d9fafe None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:24.220875 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a967d-789e-4b00-8cc2-bfdc86d9fafe None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:24.221145 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a967d-789e-4b00-8cc2-bfdc86d9fafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:24.240395 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-5e5a967d-789e-4b00-8cc2-bfdc86d9fafe None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:44:24.241614 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-5e5a967d-789e-4b00-8cc2-bfdc86d9fafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:24.300032 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1444/2846] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:44:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:44:24.307703 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc69820-89a8-46fe-8586-5a84c3cc08c2 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:24.307992 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc69820-89a8-46fe-8586-5a84c3cc08c2 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:24.308294 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc69820-89a8-46fe-8586-5a84c3cc08c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:24.328888 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-fcc69820-89a8-46fe-8586-5a84c3cc08c2 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:44:24.330100 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-fcc69820-89a8-46fe-8586-5a84c3cc08c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:24.338380 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1403/2847] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:44:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:44:24.347502 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5a967d-789e-4b00-8cc2-bfdc86d9fafe None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:24.357045 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5a967d-789e-4b00-8cc2-bfdc86d9fafe None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:24.361646 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6d31a-4558-47bd-839d-eaa5e3528a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:24.362249 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6d31a-4558-47bd-839d-eaa5e3528a4d admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:24.362522 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6d31a-4558-47bd-839d-eaa5e3528a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:24.362790 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6d31a-4558-47bd-839d-eaa5e3528a4d admin admin] PATH_INFO: `/v3/domains` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:24.363481 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e6d31a-4558-47bd-839d-eaa5e3528a4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80322) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:44:24.363778 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e6d31a-4558-47bd-839d-eaa5e3528a4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:24.365387 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e6d31a-4558-47bd-839d-eaa5e3528a4d admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:24.376792 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1445/2848] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:44:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:44:24.381123 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc69820-89a8-46fe-8586-5a84c3cc08c2 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:24.390354 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc69820-89a8-46fe-8586-5a84c3cc08c2 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:24.396080 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb3d3a-53d8-42d3-884a-331b740671fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:24.400836 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb3d3a-53d8-42d3-884a-331b740671fa admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:24.400836 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb3d3a-53d8-42d3-884a-331b740671fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:24.400836 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb3d3a-53d8-42d3-884a-331b740671fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:24.400836 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eb3d3a-53d8-42d3-884a-331b740671fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:24.400836 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eb3d3a-53d8-42d3-884a-331b740671fa admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:24.499159 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1404/2849] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:44:24 2022] POST /identity/v3/projects => generated 387 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:44:24.503437 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6d31a-4558-47bd-839d-eaa5e3528a4d admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:24.513372 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6d31a-4558-47bd-839d-eaa5e3528a4d admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:24.570496 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd234ed8-7c2b-4532-a7c3-43fe72276b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:24.571041 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd234ed8-7c2b-4532-a7c3-43fe72276b52 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:24.571372 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd234ed8-7c2b-4532-a7c3-43fe72276b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:24.571648 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd234ed8-7c2b-4532-a7c3-43fe72276b52 admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:24.572375 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd234ed8-7c2b-4532-a7c3-43fe72276b52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:24.574369 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd234ed8-7c2b-4532-a7c3-43fe72276b52 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:24.656255 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1446/2850] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:44:24 2022] POST /identity/v3/users => generated 294 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:44:24.660798 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb3d3a-53d8-42d3-884a-331b740671fa admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:24.671060 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb3d3a-53d8-42d3-884a-331b740671fa admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:24.676340 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e058d0e6-4598-41c5-ad03-87661a01823b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:24.676977 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058d0e6-4598-41c5-ad03-87661a01823b admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:24.677255 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058d0e6-4598-41c5-ad03-87661a01823b admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:24.677527 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058d0e6-4598-41c5-ad03-87661a01823b admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:24.678238 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e058d0e6-4598-41c5-ad03-87661a01823b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:24.679846 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e058d0e6-4598-41c5-ad03-87661a01823b admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:24.689781 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1405/2851] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:44:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:44:24.693585 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd234ed8-7c2b-4532-a7c3-43fe72276b52 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:24.703491 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd234ed8-7c2b-4532-a7c3-43fe72276b52 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:24.708146 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0710ca-c217-4e55-bc0d-26f9c3ef490a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:24.708769 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0710ca-c217-4e55-bc0d-26f9c3ef490a admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:24.709044 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0710ca-c217-4e55-bc0d-26f9c3ef490a admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:24.709310 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0710ca-c217-4e55-bc0d-26f9c3ef490a admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:24.709984 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0710ca-c217-4e55-bc0d-26f9c3ef490a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:24.711539 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0710ca-c217-4e55-bc0d-26f9c3ef490a admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:24.717134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:44:24.723682 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1447/2852] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:44:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:44:24.729045 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e058d0e6-4598-41c5-ad03-87661a01823b admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:24.739640 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e058d0e6-4598-41c5-ad03-87661a01823b admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:24.744279 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b4600-8761-432d-bc7e-94d5b6d38dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:24.744973 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b4600-8761-432d-bc7e-94d5b6d38dee admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:24.745249 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b4600-8761-432d-bc7e-94d5b6d38dee admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:24.745520 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b4600-8761-432d-bc7e-94d5b6d38dee admin admin] PATH_INFO: `/v3/projects/b3485dcdf1324cda91ea6ae10376e9a3/users/9255571cbd4b4942b13bfb2b7ea322ce/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:24.759766 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b4600-8761-432d-bc7e-94d5b6d38dee admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3485dcdf1324cda91ea6ae10376e9a3, user_id=9255571cbd4b4942b13bfb2b7ea322ce, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:24.762512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b4600-8761-432d-bc7e-94d5b6d38dee admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:24.840257 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1406/2853] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:44:24 2022] PUT /identity/v3/projects/b3485dcdf1324cda91ea6ae10376e9a3/users/9255571cbd4b4942b13bfb2b7ea322ce/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:44:24.851377 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1134b02e-6968-4169-96b8-aa8adf99002a None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:24.851875 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1134b02e-6968-4169-96b8-aa8adf99002a None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:24.851993 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1134b02e-6968-4169-96b8-aa8adf99002a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:24.880943 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-1134b02e-6968-4169-96b8-aa8adf99002a None None] MFA Rules not processed for user `9255571cbd4b4942b13bfb2b7ea322ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:44:24.882153 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-1134b02e-6968-4169-96b8-aa8adf99002a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:24.944957 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1448/2854] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:44:24 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:44:24.949386 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b4600-8761-432d-bc7e-94d5b6d38dee admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:25.011030 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b4600-8761-432d-bc7e-94d5b6d38dee admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:25.015583 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b46c2c-4c5c-4b7b-a623-850f6da92730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:25.017131 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b46c2c-4c5c-4b7b-a623-850f6da92730 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:25.017406 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b46c2c-4c5c-4b7b-a623-850f6da92730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:25.017669 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b46c2c-4c5c-4b7b-a623-850f6da92730 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:25.018313 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b46c2c-4c5c-4b7b-a623-850f6da92730 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:25.020177 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b46c2c-4c5c-4b7b-a623-850f6da92730 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:25.108810 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1407/2855] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:44:24 2022] POST /identity/v3/projects => generated 389 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:44:25.113377 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1134b02e-6968-4169-96b8-aa8adf99002a None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:25.123953 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1134b02e-6968-4169-96b8-aa8adf99002a None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:25.178866 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a1314b-f308-44d9-a859-d91b9d63d8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:25.179533 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1314b-f308-44d9-a859-d91b9d63d8ff admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:25.179809 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1314b-f308-44d9-a859-d91b9d63d8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:25.180076 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1314b-f308-44d9-a859-d91b9d63d8ff admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:25.180761 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a1314b-f308-44d9-a859-d91b9d63d8ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:25.182640 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a1314b-f308-44d9-a859-d91b9d63d8ff admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:25.262723 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1449/2856] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:44:25 2022] POST /identity/v3/users => generated 301 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:44:25.266686 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b46c2c-4c5c-4b7b-a623-850f6da92730 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:25.277099 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b46c2c-4c5c-4b7b-a623-850f6da92730 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:25.282214 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba25b39-8bf2-42f7-875c-1a569c8f3b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:25.282841 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba25b39-8bf2-42f7-875c-1a569c8f3b2a admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:25.283238 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba25b39-8bf2-42f7-875c-1a569c8f3b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:25.283511 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba25b39-8bf2-42f7-875c-1a569c8f3b2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:25.284190 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba25b39-8bf2-42f7-875c-1a569c8f3b2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:25.285804 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba25b39-8bf2-42f7-875c-1a569c8f3b2a admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:25.296216 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1408/2857] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:44:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:44:25.300964 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a1314b-f308-44d9-a859-d91b9d63d8ff admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:25.311288 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a1314b-f308-44d9-a859-d91b9d63d8ff admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:25.316691 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efdef6c-01ea-44ed-b425-e194b0e19504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:25.317348 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efdef6c-01ea-44ed-b425-e194b0e19504 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:25.317626 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efdef6c-01ea-44ed-b425-e194b0e19504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:25.317927 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efdef6c-01ea-44ed-b425-e194b0e19504 admin admin] PATH_INFO: `/v3/projects/24f6ae90541b4105bc32dfdbf5b0a0e6/users/62caa5ecb53b403d92c7f28711afeb70/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:25.332405 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efdef6c-01ea-44ed-b425-e194b0e19504 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24f6ae90541b4105bc32dfdbf5b0a0e6, user_id=62caa5ecb53b403d92c7f28711afeb70, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:25.335069 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efdef6c-01ea-44ed-b425-e194b0e19504 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:25.382791 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1450/2858] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:44:25 2022] PUT /identity/v3/projects/24f6ae90541b4105bc32dfdbf5b0a0e6/users/62caa5ecb53b403d92c7f28711afeb70/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:44:25.386809 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba25b39-8bf2-42f7-875c-1a569c8f3b2a admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:25.396991 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba25b39-8bf2-42f7-875c-1a569c8f3b2a admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:25.426741 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:44:25.458359 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2317fbb-7225-459f-a6c3-cc8ab527a3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:25.459627 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2317fbb-7225-459f-a6c3-cc8ab527a3d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:25.459911 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2317fbb-7225-459f-a6c3-cc8ab527a3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:25.460187 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2317fbb-7225-459f-a6c3-cc8ab527a3d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:25.460930 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2317fbb-7225-459f-a6c3-cc8ab527a3d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:25.462891 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2317fbb-7225-459f-a6c3-cc8ab527a3d5 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:25.474519 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1409/2859] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:44:25 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:44:25.482888 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efdef6c-01ea-44ed-b425-e194b0e19504 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:25.494027 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efdef6c-01ea-44ed-b425-e194b0e19504 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:25.498675 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166826ee-bb0a-4ca2-a6d6-22cf90dc74b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:25.499329 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166826ee-bb0a-4ca2-a6d6-22cf90dc74b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:25.499606 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166826ee-bb0a-4ca2-a6d6-22cf90dc74b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:25.499885 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166826ee-bb0a-4ca2-a6d6-22cf90dc74b2 admin admin] PATH_INFO: `/v3/projects/24f6ae90541b4105bc32dfdbf5b0a0e6/users/62caa5ecb53b403d92c7f28711afeb70/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:25.502146 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166826ee-bb0a-4ca2-a6d6-22cf90dc74b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24f6ae90541b4105bc32dfdbf5b0a0e6, user_id=62caa5ecb53b403d92c7f28711afeb70, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:25.504830 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166826ee-bb0a-4ca2-a6d6-22cf90dc74b2 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:25.514809 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1451/2860] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:44:25 2022] PUT /identity/v3/projects/24f6ae90541b4105bc32dfdbf5b0a0e6/users/62caa5ecb53b403d92c7f28711afeb70/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:44:25.526374 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c992645-53f3-43bf-a093-aa4c29283f7d None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:25.526596 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c992645-53f3-43bf-a093-aa4c29283f7d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:25.526871 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c992645-53f3-43bf-a093-aa4c29283f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:25.556111 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-3c992645-53f3-43bf-a093-aa4c29283f7d None None] MFA Rules not processed for user `62caa5ecb53b403d92c7f28711afeb70`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:44:25.557341 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-3c992645-53f3-43bf-a093-aa4c29283f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:25.624570 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1410/2861] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:44:25 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:44:25.636779 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166826ee-bb0a-4ca2-a6d6-22cf90dc74b2 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:25.638386 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-166826ee-bb0a-4ca2-a6d6-22cf90dc74b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:25.685700 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:44:25.697354 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166826ee-bb0a-4ca2-a6d6-22cf90dc74b2 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:25.700960 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afa1e5-e6ce-44c0-90a1-679e612a4653 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:25.708317 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afa1e5-e6ce-44c0-90a1-679e612a4653 service cinder] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:25.708317 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afa1e5-e6ce-44c0-90a1-679e612a4653 service cinder] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:25.708317 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afa1e5-e6ce-44c0-90a1-679e612a4653 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:25.711518 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63afa1e5-e6ce-44c0-90a1-679e612a4653 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:25.713326 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63afa1e5-e6ce-44c0-90a1-679e612a4653 service cinder] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:25.723391 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1452/2862] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:44:25 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:44:25.865070 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c992645-53f3-43bf-a093-aa4c29283f7d None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:25.927599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c992645-53f3-43bf-a093-aa4c29283f7d None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:25.931918 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814afd35-6903-4258-a40a-5f08946ca335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:25.932578 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814afd35-6903-4258-a40a-5f08946ca335 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:25.932854 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814afd35-6903-4258-a40a-5f08946ca335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:25.933144 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814afd35-6903-4258-a40a-5f08946ca335 admin admin] PATH_INFO: `/v3/users/9255571cbd4b4942b13bfb2b7ea322ce` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:25.934280 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814afd35-6903-4258-a40a-5f08946ca335 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9255571cbd4b4942b13bfb2b7ea322ce)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:25.936165 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814afd35-6903-4258-a40a-5f08946ca335 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:26.097280 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-814afd35-6903-4258-a40a-5f08946ca335 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9255571cbd4b4942b13bfb2b7ea322ce'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:26.102643 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:44:26.109182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-814afd35-6903-4258-a40a-5f08946ca335 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9255571cbd4b4942b13bfb2b7ea322ce'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:26.170474 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-814afd35-6903-4258-a40a-5f08946ca335 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9255571cbd4b4942b13bfb2b7ea322ce'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:26.171091 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-814afd35-6903-4258-a40a-5f08946ca335 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9255571cbd4b4942b13bfb2b7ea322ce'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:26.178901 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-814afd35-6903-4258-a40a-5f08946ca335 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9255571cbd4b4942b13bfb2b7ea322ce'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:26.184879 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1411/2863] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:44:25 2022] DELETE /identity/v3/users/9255571cbd4b4942b13bfb2b7ea322ce => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:44:26.190005 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b146467-21c8-4466-93d6-4033a0668d2b None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:26.190519 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b146467-21c8-4466-93d6-4033a0668d2b None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:26.190519 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b146467-21c8-4466-93d6-4033a0668d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:26.209800 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-2b146467-21c8-4466-93d6-4033a0668d2b None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:44:26.210988 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-2b146467-21c8-4466-93d6-4033a0668d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:26.270268 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1453/2864] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:44:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:44:26.279768 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814afd35-6903-4258-a40a-5f08946ca335 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:26.281275 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-814afd35-6903-4258-a40a-5f08946ca335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:26.338211 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814afd35-6903-4258-a40a-5f08946ca335 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:26.341842 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0480c2d1-0e62-48f5-9047-5c657dfda0ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:26.342447 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480c2d1-0e62-48f5-9047-5c657dfda0ce service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:26.342721 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480c2d1-0e62-48f5-9047-5c657dfda0ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:26.342990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480c2d1-0e62-48f5-9047-5c657dfda0ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:26.351892 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0480c2d1-0e62-48f5-9047-5c657dfda0ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:26.353336 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0480c2d1-0e62-48f5-9047-5c657dfda0ce service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:26.363245 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1412/2865] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:44:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:44:26.733277 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b146467-21c8-4466-93d6-4033a0668d2b None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:26.734866 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-2b146467-21c8-4466-93d6-4033a0668d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:26.749132 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b146467-21c8-4466-93d6-4033a0668d2b None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:26.755431 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19229f1-3625-45d5-8762-f25d5e6f807b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:26.756347 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19229f1-3625-45d5-8762-f25d5e6f807b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:26.756653 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19229f1-3625-45d5-8762-f25d5e6f807b admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:26.757002 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19229f1-3625-45d5-8762-f25d5e6f807b admin admin] PATH_INFO: `/v3/projects/b3485dcdf1324cda91ea6ae10376e9a3` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:26.758092 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19229f1-3625-45d5-8762-f25d5e6f807b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3485dcdf1324cda91ea6ae10376e9a3)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:26.760157 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19229f1-3625-45d5-8762-f25d5e6f807b admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:26.868036 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-d19229f1-3625-45d5-8762-f25d5e6f807b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3485dcdf1324cda91ea6ae10376e9a3', 'request_id': 'req-d19229f1-3625-45d5-8762-f25d5e6f807b'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:26.874119 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-d19229f1-3625-45d5-8762-f25d5e6f807b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3485dcdf1324cda91ea6ae10376e9a3', 'request_id': 'req-d19229f1-3625-45d5-8762-f25d5e6f807b'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:26.880142 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-d19229f1-3625-45d5-8762-f25d5e6f807b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3485dcdf1324cda91ea6ae10376e9a3', 'request_id': 'req-d19229f1-3625-45d5-8762-f25d5e6f807b'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:26.924715 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-d19229f1-3625-45d5-8762-f25d5e6f807b admin admin] The token cache is being invalidate because project b3485dcdf1324cda91ea6ae10376e9a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:44:26.925003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-d19229f1-3625-45d5-8762-f25d5e6f807b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:26.925616 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-d19229f1-3625-45d5-8762-f25d5e6f807b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:26.927261 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1454/2866] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:44:26 2022] DELETE /identity/v3/projects/b3485dcdf1324cda91ea6ae10376e9a3 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:44:26.931257 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0480c2d1-0e62-48f5-9047-5c657dfda0ce service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:26.941558 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0480c2d1-0e62-48f5-9047-5c657dfda0ce service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:26.996110 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd21538-e33f-4338-aaac-ac9fe7177a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:26.996772 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd21538-e33f-4338-aaac-ac9fe7177a80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:26.997050 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd21538-e33f-4338-aaac-ac9fe7177a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:26.997322 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd21538-e33f-4338-aaac-ac9fe7177a80 admin admin] PATH_INFO: `/v3/users/62caa5ecb53b403d92c7f28711afeb70` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:26.998466 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd21538-e33f-4338-aaac-ac9fe7177a80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62caa5ecb53b403d92c7f28711afeb70)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:27.000369 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd21538-e33f-4338-aaac-ac9fe7177a80 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:27.183915 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-3fd21538-e33f-4338-aaac-ac9fe7177a80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62caa5ecb53b403d92c7f28711afeb70'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:27.196736 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-3fd21538-e33f-4338-aaac-ac9fe7177a80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62caa5ecb53b403d92c7f28711afeb70'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:27.251824 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-3fd21538-e33f-4338-aaac-ac9fe7177a80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62caa5ecb53b403d92c7f28711afeb70'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:27.252396 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-3fd21538-e33f-4338-aaac-ac9fe7177a80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62caa5ecb53b403d92c7f28711afeb70'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:27.257159 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:44:27.259876 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-3fd21538-e33f-4338-aaac-ac9fe7177a80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62caa5ecb53b403d92c7f28711afeb70'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:27.266981 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1413/2867] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:44:26 2022] DELETE /identity/v3/users/62caa5ecb53b403d92c7f28711afeb70 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:44:27.682885 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19229f1-3625-45d5-8762-f25d5e6f807b admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:27.684587 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-d19229f1-3625-45d5-8762-f25d5e6f807b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:27.697473 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19229f1-3625-45d5-8762-f25d5e6f807b admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:27.755487 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b5c82e-432e-4522-9dec-669ad55d71a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:27.756046 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5c82e-432e-4522-9dec-669ad55d71a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:27.756321 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5c82e-432e-4522-9dec-669ad55d71a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:27.756618 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5c82e-432e-4522-9dec-669ad55d71a1 admin admin] PATH_INFO: `/v3/projects/24f6ae90541b4105bc32dfdbf5b0a0e6` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:27.757754 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b5c82e-432e-4522-9dec-669ad55d71a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24f6ae90541b4105bc32dfdbf5b0a0e6)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:27.759646 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b5c82e-432e-4522-9dec-669ad55d71a1 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:27.770068 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:44:27.863944 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-09b5c82e-432e-4522-9dec-669ad55d71a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24f6ae90541b4105bc32dfdbf5b0a0e6', 'request_id': 'req-09b5c82e-432e-4522-9dec-669ad55d71a1'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:27.869967 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-09b5c82e-432e-4522-9dec-669ad55d71a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24f6ae90541b4105bc32dfdbf5b0a0e6', 'request_id': 'req-09b5c82e-432e-4522-9dec-669ad55d71a1'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:27.875267 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-09b5c82e-432e-4522-9dec-669ad55d71a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24f6ae90541b4105bc32dfdbf5b0a0e6', 'request_id': 'req-09b5c82e-432e-4522-9dec-669ad55d71a1'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:27.919245 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-09b5c82e-432e-4522-9dec-669ad55d71a1 admin admin] The token cache is being invalidate because project 24f6ae90541b4105bc32dfdbf5b0a0e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:44:27.919525 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-09b5c82e-432e-4522-9dec-669ad55d71a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:27.920163 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-09b5c82e-432e-4522-9dec-669ad55d71a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:27.921793 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1455/2868] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:44:27 2022] DELETE /identity/v3/projects/24f6ae90541b4105bc32dfdbf5b0a0e6 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:44:27.927175 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1483a947-2e87-492a-b9ac-79094107a51d None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:27.927655 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1483a947-2e87-492a-b9ac-79094107a51d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:27.927776 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1483a947-2e87-492a-b9ac-79094107a51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:27.947573 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-1483a947-2e87-492a-b9ac-79094107a51d None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:44:27.948793 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-1483a947-2e87-492a-b9ac-79094107a51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:28.007756 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1414/2869] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:44:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:44:28.015486 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9ba72-46da-4e17-adb9-68aeb9494024 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:28.015773 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9ba72-46da-4e17-adb9-68aeb9494024 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:28.016050 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9ba72-46da-4e17-adb9-68aeb9494024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:28.037003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-01a9ba72-46da-4e17-adb9-68aeb9494024 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:44:28.038218 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-01a9ba72-46da-4e17-adb9-68aeb9494024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:28.046239 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1456/2870] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:44:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:44:28.050563 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1483a947-2e87-492a-b9ac-79094107a51d None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:28.060012 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1483a947-2e87-492a-b9ac-79094107a51d None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:28.064654 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ca7db-2eef-46f2-bf9d-7b74abcac456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:28.065259 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ca7db-2eef-46f2-bf9d-7b74abcac456 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:28.065536 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ca7db-2eef-46f2-bf9d-7b74abcac456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:28.065807 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ca7db-2eef-46f2-bf9d-7b74abcac456 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:28.066497 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29ca7db-2eef-46f2-bf9d-7b74abcac456 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:44:28.066799 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29ca7db-2eef-46f2-bf9d-7b74abcac456 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:28.068372 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29ca7db-2eef-46f2-bf9d-7b74abcac456 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:28.080248 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1415/2871] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:44:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 14:44:28.084359 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a9ba72-46da-4e17-adb9-68aeb9494024 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:28.093742 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a9ba72-46da-4e17-adb9-68aeb9494024 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:28.099218 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a04be-3e10-4abc-848f-8f2bc2d6adf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:28.099817 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a04be-3e10-4abc-848f-8f2bc2d6adf9 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:28.100098 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a04be-3e10-4abc-848f-8f2bc2d6adf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:28.100402 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a04be-3e10-4abc-848f-8f2bc2d6adf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:28.101095 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477a04be-3e10-4abc-848f-8f2bc2d6adf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:28.102984 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477a04be-3e10-4abc-848f-8f2bc2d6adf9 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:28.178304 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1457/2872] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:44:28 2022] POST /identity/v3/projects => generated 371 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:44:28.182345 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ca7db-2eef-46f2-bf9d-7b74abcac456 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:28.192785 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ca7db-2eef-46f2-bf9d-7b74abcac456 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:28.249396 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909eaea-f9bb-4af1-862c-70f31be5b8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:28.250006 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909eaea-f9bb-4af1-862c-70f31be5b8c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:28.250283 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909eaea-f9bb-4af1-862c-70f31be5b8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:28.250557 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909eaea-f9bb-4af1-862c-70f31be5b8c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:28.251727 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6909eaea-f9bb-4af1-862c-70f31be5b8c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:28.253637 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6909eaea-f9bb-4af1-862c-70f31be5b8c3 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:28.308454 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1416/2873] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:44:28 2022] POST /identity/v3/users => generated 286 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:44:28.312597 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a04be-3e10-4abc-848f-8f2bc2d6adf9 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:28.323939 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a04be-3e10-4abc-848f-8f2bc2d6adf9 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:28.328617 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f5eb07-5a14-4a1a-8f1c-c84d0a27c962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:28.329939 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5eb07-5a14-4a1a-8f1c-c84d0a27c962 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:28.330334 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5eb07-5a14-4a1a-8f1c-c84d0a27c962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:28.330613 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5eb07-5a14-4a1a-8f1c-c84d0a27c962 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:28.331332 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f5eb07-5a14-4a1a-8f1c-c84d0a27c962 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:28.332983 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f5eb07-5a14-4a1a-8f1c-c84d0a27c962 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:28.343140 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1458/2874] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:44:28.347872 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909eaea-f9bb-4af1-862c-70f31be5b8c3 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:28.357786 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909eaea-f9bb-4af1-862c-70f31be5b8c3 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:28.363066 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf42a5-9b37-49d1-a012-cf80fd4e1922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:28.363657 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf42a5-9b37-49d1-a012-cf80fd4e1922 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:28.363933 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf42a5-9b37-49d1-a012-cf80fd4e1922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:28.364204 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf42a5-9b37-49d1-a012-cf80fd4e1922 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:28.364937 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cf42a5-9b37-49d1-a012-cf80fd4e1922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:28.366517 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cf42a5-9b37-49d1-a012-cf80fd4e1922 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:28.376512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1417/2875] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:44:28.380700 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f5eb07-5a14-4a1a-8f1c-c84d0a27c962 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:28.390839 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f5eb07-5a14-4a1a-8f1c-c84d0a27c962 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:28.395852 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bcd85c-6416-44d3-b4a4-2bfc3b5fe950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:28.396554 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcd85c-6416-44d3-b4a4-2bfc3b5fe950 admin admin] REQUEST_METHOD: `PUT` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:28.396839 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcd85c-6416-44d3-b4a4-2bfc3b5fe950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:28.397125 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcd85c-6416-44d3-b4a4-2bfc3b5fe950 admin admin] PATH_INFO: `/v3/projects/2c2a87c16dd04020b6d13a9eff54ae0c/users/38af11bd133346f88da5b5b21184579d/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:28.412389 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bcd85c-6416-44d3-b4a4-2bfc3b5fe950 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c2a87c16dd04020b6d13a9eff54ae0c, user_id=38af11bd133346f88da5b5b21184579d, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:28.415072 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bcd85c-6416-44d3-b4a4-2bfc3b5fe950 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:28.485933 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1459/2876] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:44:28 2022] PUT /identity/v3/projects/2c2a87c16dd04020b6d13a9eff54ae0c/users/38af11bd133346f88da5b5b21184579d/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 14:44:28.496236 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6114ac3-e99b-44b2-b16e-c016b52a11c6 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:28.496546 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6114ac3-e99b-44b2-b16e-c016b52a11c6 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:28.496816 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6114ac3-e99b-44b2-b16e-c016b52a11c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:28.527598 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-b6114ac3-e99b-44b2-b16e-c016b52a11c6 None None] MFA Rules not processed for user `38af11bd133346f88da5b5b21184579d`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:44:28.528984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-b6114ac3-e99b-44b2-b16e-c016b52a11c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:28.584242 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:44:28.600559 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1418/2877] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:44:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:44:28.604441 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bcd85c-6416-44d3-b4a4-2bfc3b5fe950 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:28.673683 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bcd85c-6416-44d3-b4a4-2bfc3b5fe950 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:28.678551 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee53539-01d5-46f1-9c63-cc0e529375d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:28.679279 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee53539-01d5-46f1-9c63-cc0e529375d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:28.679559 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee53539-01d5-46f1-9c63-cc0e529375d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:28.679839 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee53539-01d5-46f1-9c63-cc0e529375d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:28.680537 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee53539-01d5-46f1-9c63-cc0e529375d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:28.682540 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee53539-01d5-46f1-9c63-cc0e529375d5 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:28.773292 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1460/2878] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:44:28 2022] POST /identity/v3/projects => generated 375 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:44:28.777416 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6114ac3-e99b-44b2-b16e-c016b52a11c6 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:28.788294 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6114ac3-e99b-44b2-b16e-c016b52a11c6 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:28.846010 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b821115-0959-415c-bee4-24bffa4f69e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:28.846550 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b821115-0959-415c-bee4-24bffa4f69e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:28.846824 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b821115-0959-415c-bee4-24bffa4f69e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:28.847094 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b821115-0959-415c-bee4-24bffa4f69e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:28.847758 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b821115-0959-415c-bee4-24bffa4f69e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:28.849690 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b821115-0959-415c-bee4-24bffa4f69e5 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:28.937453 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1419/2879] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:44:28 2022] POST /identity/v3/users => generated 294 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:44:28.942041 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee53539-01d5-46f1-9c63-cc0e529375d5 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:28.952487 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee53539-01d5-46f1-9c63-cc0e529375d5 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:28.957490 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18886c8-abde-4a25-91bb-e4e2f0b1bbf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:28.958096 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18886c8-abde-4a25-91bb-e4e2f0b1bbf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:28.958376 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18886c8-abde-4a25-91bb-e4e2f0b1bbf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:28.958648 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18886c8-abde-4a25-91bb-e4e2f0b1bbf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:28.959334 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18886c8-abde-4a25-91bb-e4e2f0b1bbf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:28.960986 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18886c8-abde-4a25-91bb-e4e2f0b1bbf0 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:28.972671 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1461/2880] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:44:28.976522 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b821115-0959-415c-bee4-24bffa4f69e5 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:28.987223 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b821115-0959-415c-bee4-24bffa4f69e5 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:28.991887 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fcb91f-2a77-4eb0-90d8-a25a03fd62a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:28.993349 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fcb91f-2a77-4eb0-90d8-a25a03fd62a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:28.993349 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fcb91f-2a77-4eb0-90d8-a25a03fd62a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:28.993349 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fcb91f-2a77-4eb0-90d8-a25a03fd62a5 admin admin] PATH_INFO: `/v3/projects/69a6c02fcaa144419a662f58ec3f1865/users/d6f399eb46f2454f8722512be8676f6c/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:29.008748 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fcb91f-2a77-4eb0-90d8-a25a03fd62a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69a6c02fcaa144419a662f58ec3f1865, user_id=d6f399eb46f2454f8722512be8676f6c, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:29.012137 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fcb91f-2a77-4eb0-90d8-a25a03fd62a5 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:29.097131 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1420/2881] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:44:28 2022] PUT /identity/v3/projects/69a6c02fcaa144419a662f58ec3f1865/users/d6f399eb46f2454f8722512be8676f6c/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:44:29.101285 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18886c8-abde-4a25-91bb-e4e2f0b1bbf0 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:29.111723 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18886c8-abde-4a25-91bb-e4e2f0b1bbf0 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:29.116292 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:44:29.169482 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f7ac7e-31e9-4211-874c-2a3ec8794aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:29.170110 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7ac7e-31e9-4211-874c-2a3ec8794aa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:29.170370 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7ac7e-31e9-4211-874c-2a3ec8794aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:29.170640 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7ac7e-31e9-4211-874c-2a3ec8794aa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:29.171328 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f7ac7e-31e9-4211-874c-2a3ec8794aa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:29.173030 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f7ac7e-31e9-4211-874c-2a3ec8794aa2 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:29.186316 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1462/2882] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:44:29.189182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fcb91f-2a77-4eb0-90d8-a25a03fd62a5 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:29.198834 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fcb91f-2a77-4eb0-90d8-a25a03fd62a5 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:29.203538 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c833b78-6127-497b-a20f-51f7331fe9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:29.204844 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c833b78-6127-497b-a20f-51f7331fe9d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:29.205127 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c833b78-6127-497b-a20f-51f7331fe9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:29.205398 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c833b78-6127-497b-a20f-51f7331fe9d4 admin admin] PATH_INFO: `/v3/projects/69a6c02fcaa144419a662f58ec3f1865/users/d6f399eb46f2454f8722512be8676f6c/roles/299c794c7acb4a5ca36288c3191aad10` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:29.207403 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c833b78-6127-497b-a20f-51f7331fe9d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69a6c02fcaa144419a662f58ec3f1865, user_id=d6f399eb46f2454f8722512be8676f6c, role_id=299c794c7acb4a5ca36288c3191aad10)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:29.210042 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c833b78-6127-497b-a20f-51f7331fe9d4 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:29.221721 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1421/2883] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:44:29 2022] PUT /identity/v3/projects/69a6c02fcaa144419a662f58ec3f1865/users/d6f399eb46f2454f8722512be8676f6c/roles/299c794c7acb4a5ca36288c3191aad10 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:44:29.289005 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa5e895-5d46-442b-ab53-13a284da0473 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:29.289282 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa5e895-5d46-442b-ab53-13a284da0473 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:29.289546 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa5e895-5d46-442b-ab53-13a284da0473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:29.317874 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-4aa5e895-5d46-442b-ab53-13a284da0473 None None] MFA Rules not processed for user `d6f399eb46f2454f8722512be8676f6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:44:29.319133 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-4aa5e895-5d46-442b-ab53-13a284da0473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:29.390111 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1463/2884] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:44:29 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:44:29.401772 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c833b78-6127-497b-a20f-51f7331fe9d4 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:29.403498 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5c833b78-6127-497b-a20f-51f7331fe9d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:29.461623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c833b78-6127-497b-a20f-51f7331fe9d4 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:29.465789 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b2bbe-2c19-40af-ab43-68e8ecca2ba8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88d6a1bae9834e2e91d4ab3a4c00959b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:29.466400 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b2bbe-2c19-40af-ab43-68e8ecca2ba8 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:29.466673 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b2bbe-2c19-40af-ab43-68e8ecca2ba8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:29.466993 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b2bbe-2c19-40af-ab43-68e8ecca2ba8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:29.526176 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:44:29.528586 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b2bbe-2c19-40af-ab43-68e8ecca2ba8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:29.530119 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b2bbe-2c19-40af-ab43-68e8ecca2ba8 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:29.539939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1422/2885] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:44:29 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:44:34.538285 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad2682-435c-474f-9048-3fa115b6c13c None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:34.538788 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad2682-435c-474f-9048-3fa115b6c13c None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:34.538898 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad2682-435c-474f-9048-3fa115b6c13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:34.558027 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-e4ad2682-435c-474f-9048-3fa115b6c13c None None] MFA Rules not processed for user `f7b9ccd1b8cc4832a9c77513b5f21815`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:44:34.559256 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-e4ad2682-435c-474f-9048-3fa115b6c13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:34.613159 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1464/2886] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:44:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:44:34.944354 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b2bbe-2c19-40af-ab43-68e8ecca2ba8 service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:34.945898 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-af3b2bbe-2c19-40af-ab43-68e8ecca2ba8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:35.003119 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b2bbe-2c19-40af-ab43-68e8ecca2ba8 service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:35.006932 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d54c3-4e05-44b7-a337-2d5ae8f2e5c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:35.007542 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d54c3-4e05-44b7-a337-2d5ae8f2e5c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:35.007820 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d54c3-4e05-44b7-a337-2d5ae8f2e5c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:35.008093 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d54c3-4e05-44b7-a337-2d5ae8f2e5c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:35.009999 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-b58d54c3-4e05-44b7-a337-2d5ae8f2e5c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:35.019853 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d54c3-4e05-44b7-a337-2d5ae8f2e5c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:35.021402 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d54c3-4e05-44b7-a337-2d5ae8f2e5c5 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:35.029806 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1423/2887] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:44:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:44:35.724668 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ad2682-435c-474f-9048-3fa115b6c13c None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:35.763627 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=100 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:44:35.789822 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ad2682-435c-474f-9048-3fa115b6c13c None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:35.791299 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e2a0c6-12a0-4cc9-9275-a45003b21bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:35.791931 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2a0c6-12a0-4cc9-9275-a45003b21bef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:35.792213 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2a0c6-12a0-4cc9-9275-a45003b21bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:35.792518 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2a0c6-12a0-4cc9-9275-a45003b21bef admin admin] PATH_INFO: `/v3/users/38af11bd133346f88da5b5b21184579d` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:35.793679 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e2a0c6-12a0-4cc9-9275-a45003b21bef admin admin] RBAC: Authorizing `identity:delete_user(user_id=38af11bd133346f88da5b5b21184579d)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:35.795597 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e2a0c6-12a0-4cc9-9275-a45003b21bef admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:35.945470 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-86e2a0c6-12a0-4cc9-9275-a45003b21bef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38af11bd133346f88da5b5b21184579d'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:35.952589 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-86e2a0c6-12a0-4cc9-9275-a45003b21bef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38af11bd133346f88da5b5b21184579d'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:35.953272 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-86e2a0c6-12a0-4cc9-9275-a45003b21bef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38af11bd133346f88da5b5b21184579d'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:35.953872 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-86e2a0c6-12a0-4cc9-9275-a45003b21bef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38af11bd133346f88da5b5b21184579d'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:35.966251 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-86e2a0c6-12a0-4cc9-9275-a45003b21bef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38af11bd133346f88da5b5b21184579d'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:36.049810 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1465/2888] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:44:35 2022] DELETE /identity/v3/users/38af11bd133346f88da5b5b21184579d => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:44:36.054937 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234c6b3-ec30-4da4-96bd-c4251d741747 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:36.059177 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234c6b3-ec30-4da4-96bd-c4251d741747 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:36.059177 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234c6b3-ec30-4da4-96bd-c4251d741747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:36.075193 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-c234c6b3-ec30-4da4-96bd-c4251d741747 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:44:36.076372 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c234c6b3-ec30-4da4-96bd-c4251d741747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:36.134512 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1424/2889] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:44:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:44:36.144271 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e2a0c6-12a0-4cc9-9275-a45003b21bef admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:36.145716 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-86e2a0c6-12a0-4cc9-9275-a45003b21bef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:36.201779 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e2a0c6-12a0-4cc9-9275-a45003b21bef admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:36.205327 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfaca6c-e8b6-403e-bb27-8d3cd73c3d8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:36.205938 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfaca6c-e8b6-403e-bb27-8d3cd73c3d8e service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:36.206256 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfaca6c-e8b6-403e-bb27-8d3cd73c3d8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:36.206527 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfaca6c-e8b6-403e-bb27-8d3cd73c3d8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:36.215571 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfaca6c-e8b6-403e-bb27-8d3cd73c3d8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:36.217014 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfaca6c-e8b6-403e-bb27-8d3cd73c3d8e service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:36.226214 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1466/2890] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:44:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:44:36.641422 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234c6b3-ec30-4da4-96bd-c4251d741747 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:36.642950 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c234c6b3-ec30-4da4-96bd-c4251d741747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:36.655940 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234c6b3-ec30-4da4-96bd-c4251d741747 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:36.660472 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c93325-7f8e-4148-bfdd-d5229e21f566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:36.661171 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c93325-7f8e-4148-bfdd-d5229e21f566 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:36.661449 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c93325-7f8e-4148-bfdd-d5229e21f566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:36.661719 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c93325-7f8e-4148-bfdd-d5229e21f566 admin admin] PATH_INFO: `/v3/projects/2c2a87c16dd04020b6d13a9eff54ae0c` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:36.662879 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c93325-7f8e-4148-bfdd-d5229e21f566 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c2a87c16dd04020b6d13a9eff54ae0c)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:36.664850 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c93325-7f8e-4148-bfdd-d5229e21f566 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:36.775721 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a7c93325-7f8e-4148-bfdd-d5229e21f566 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c2a87c16dd04020b6d13a9eff54ae0c', 'request_id': 'req-a7c93325-7f8e-4148-bfdd-d5229e21f566'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:36.785347 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a7c93325-7f8e-4148-bfdd-d5229e21f566 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c2a87c16dd04020b6d13a9eff54ae0c', 'request_id': 'req-a7c93325-7f8e-4148-bfdd-d5229e21f566'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:36.861801 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a7c93325-7f8e-4148-bfdd-d5229e21f566 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c2a87c16dd04020b6d13a9eff54ae0c', 'request_id': 'req-a7c93325-7f8e-4148-bfdd-d5229e21f566'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:36.874394 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a7c93325-7f8e-4148-bfdd-d5229e21f566 admin admin] The token cache is being invalidate because project 2c2a87c16dd04020b6d13a9eff54ae0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:44:36.874697 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a7c93325-7f8e-4148-bfdd-d5229e21f566 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:36.875334 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a7c93325-7f8e-4148-bfdd-d5229e21f566 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:36.877122 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1425/2891] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:44:36 2022] DELETE /identity/v3/projects/2c2a87c16dd04020b6d13a9eff54ae0c => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:44:36.882167 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfaca6c-e8b6-403e-bb27-8d3cd73c3d8e service neutron] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:36.892369 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfaca6c-e8b6-403e-bb27-8d3cd73c3d8e service neutron] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:36.945394 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:44:36.951689 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cfb328-b4a4-47df-9023-a1357daccc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:36.952349 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfb328-b4a4-47df-9023-a1357daccc72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:36.952687 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfb328-b4a4-47df-9023-a1357daccc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:36.952940 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfb328-b4a4-47df-9023-a1357daccc72 admin admin] PATH_INFO: `/v3/users/d6f399eb46f2454f8722512be8676f6c` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:36.954087 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cfb328-b4a4-47df-9023-a1357daccc72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6f399eb46f2454f8722512be8676f6c)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:36.955983 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cfb328-b4a4-47df-9023-a1357daccc72 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:37.119895 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-62cfb328-b4a4-47df-9023-a1357daccc72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6f399eb46f2454f8722512be8676f6c'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:37.127667 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-62cfb328-b4a4-47df-9023-a1357daccc72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6f399eb46f2454f8722512be8676f6c'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:37.128881 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-62cfb328-b4a4-47df-9023-a1357daccc72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6f399eb46f2454f8722512be8676f6c'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:37.129571 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-62cfb328-b4a4-47df-9023-a1357daccc72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6f399eb46f2454f8722512be8676f6c'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:37.143583 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-62cfb328-b4a4-47df-9023-a1357daccc72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6f399eb46f2454f8722512be8676f6c'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:37.199781 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1467/2892] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:44:36 2022] DELETE /identity/v3/users/d6f399eb46f2454f8722512be8676f6c => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:44:37.596004 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c93325-7f8e-4148-bfdd-d5229e21f566 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:37.598629 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a7c93325-7f8e-4148-bfdd-d5229e21f566 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:37.613653 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c93325-7f8e-4148-bfdd-d5229e21f566 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:37.635199 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:44:37.679033 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba53743-f0e3-4ef2-92ef-98631c5bf6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:37.679742 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba53743-f0e3-4ef2-92ef-98631c5bf6f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:37.680088 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba53743-f0e3-4ef2-92ef-98631c5bf6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:37.680359 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba53743-f0e3-4ef2-92ef-98631c5bf6f5 admin admin] PATH_INFO: `/v3/projects/69a6c02fcaa144419a662f58ec3f1865` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:37.681667 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba53743-f0e3-4ef2-92ef-98631c5bf6f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69a6c02fcaa144419a662f58ec3f1865)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:37.683985 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba53743-f0e3-4ef2-92ef-98631c5bf6f5 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:37.854459 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-fba53743-f0e3-4ef2-92ef-98631c5bf6f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69a6c02fcaa144419a662f58ec3f1865', 'request_id': 'req-fba53743-f0e3-4ef2-92ef-98631c5bf6f5'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:37.861114 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-fba53743-f0e3-4ef2-92ef-98631c5bf6f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69a6c02fcaa144419a662f58ec3f1865', 'request_id': 'req-fba53743-f0e3-4ef2-92ef-98631c5bf6f5'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:37.913002 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cfb328-b4a4-47df-9023-a1357daccc72 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:37.914236 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-fba53743-f0e3-4ef2-92ef-98631c5bf6f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69a6c02fcaa144419a662f58ec3f1865', 'request_id': 'req-fba53743-f0e3-4ef2-92ef-98631c5bf6f5'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:37.914468 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-62cfb328-b4a4-47df-9023-a1357daccc72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:37.927182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-fba53743-f0e3-4ef2-92ef-98631c5bf6f5 admin admin] The token cache is being invalidate because project 69a6c02fcaa144419a662f58ec3f1865 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:44:37.930050 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-fba53743-f0e3-4ef2-92ef-98631c5bf6f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:37.930050 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-fba53743-f0e3-4ef2-92ef-98631c5bf6f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:44:37.930050 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1426/2893] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:44:37 2022] DELETE /identity/v3/projects/69a6c02fcaa144419a662f58ec3f1865 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:44:37.934599 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da12023-5662-4181-9263-a0802fc3d8a9 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:37.934890 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da12023-5662-4181-9263-a0802fc3d8a9 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:37.935162 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da12023-5662-4181-9263-a0802fc3d8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:37.956925 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-5da12023-5662-4181-9263-a0802fc3d8a9 None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:44:37.958129 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5da12023-5662-4181-9263-a0802fc3d8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:38.009396 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cfb328-b4a4-47df-9023-a1357daccc72 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:38.017141 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1427/2894] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:44:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:44:38.025320 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec1036d-4e8e-427b-86ef-53876fc083db None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:38.025605 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec1036d-4e8e-427b-86ef-53876fc083db None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:38.025878 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec1036d-4e8e-427b-86ef-53876fc083db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:38.045020 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-3ec1036d-4e8e-427b-86ef-53876fc083db None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:44:38.046243 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-3ec1036d-4e8e-427b-86ef-53876fc083db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:38.054897 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1428/2895] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:44:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 14:44:38.059035 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec1036d-4e8e-427b-86ef-53876fc083db None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:38.061814 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:44:38.068520 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec1036d-4e8e-427b-86ef-53876fc083db None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:38.069827 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6619a9a5-9f69-4029-99be-c92e376a0589 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff761f6f81a46098f94bd2ec2be9c1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:38.070448 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619a9a5-9f69-4029-99be-c92e376a0589 service placement] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:38.070725 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619a9a5-9f69-4029-99be-c92e376a0589 service placement] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:38.070995 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619a9a5-9f69-4029-99be-c92e376a0589 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:38.072727 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-6619a9a5-9f69-4029-99be-c92e376a0589 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:38.073208 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66844bf6-3f2d-48f5-b6b9-07501ebe5343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:38.073800 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66844bf6-3f2d-48f5-b6b9-07501ebe5343 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:38.074075 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66844bf6-3f2d-48f5-b6b9-07501ebe5343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:38.074344 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66844bf6-3f2d-48f5-b6b9-07501ebe5343 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:38.075080 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66844bf6-3f2d-48f5-b6b9-07501ebe5343 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 14:44:38.075379 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66844bf6-3f2d-48f5-b6b9-07501ebe5343 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:38.076983 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66844bf6-3f2d-48f5-b6b9-07501ebe5343 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:38.084524 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6619a9a5-9f69-4029-99be-c92e376a0589 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:38.086100 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6619a9a5-9f69-4029-99be-c92e376a0589 service placement] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:38.089456 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1429/2896] 10.222.0.26 () {58 vars in 1153 bytes} [Fri Aug 5 14:44:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 14:44:38.093695 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66844bf6-3f2d-48f5-b6b9-07501ebe5343 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:38.097013 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1468/2897] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:44:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 14:44:38.103516 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66844bf6-3f2d-48f5-b6b9-07501ebe5343 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:38.108079 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12e31e4-4571-479e-8e0a-124c1debbda0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:38.108717 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e31e4-4571-479e-8e0a-124c1debbda0 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:38.109084 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e31e4-4571-479e-8e0a-124c1debbda0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:38.109262 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e31e4-4571-479e-8e0a-124c1debbda0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:38.109911 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12e31e4-4571-479e-8e0a-124c1debbda0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:38.111769 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12e31e4-4571-479e-8e0a-124c1debbda0 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:38.235524 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1430/2898] 10.222.0.26 () {60 vars in 1153 bytes} [Fri Aug 5 14:44:38 2022] POST /identity/v3/projects => generated 369 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 14:44:38.240745 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6619a9a5-9f69-4029-99be-c92e376a0589 service placement] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:38.251134 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6619a9a5-9f69-4029-99be-c92e376a0589 service placement] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:38.308716 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae726ada-59a7-4918-8d09-00a671077787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:38.309330 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae726ada-59a7-4918-8d09-00a671077787 admin admin] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:38.309610 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae726ada-59a7-4918-8d09-00a671077787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:38.309884 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae726ada-59a7-4918-8d09-00a671077787 admin admin] PATH_INFO: `/v3/users` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:38.310556 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae726ada-59a7-4918-8d09-00a671077787 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:38.312579 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae726ada-59a7-4918-8d09-00a671077787 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:38.435169 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1469/2899] 10.222.0.26 () {60 vars in 1144 bytes} [Fri Aug 5 14:44:38 2022] POST /identity/v3/users => generated 285 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 14:44:38.439300 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12e31e4-4571-479e-8e0a-124c1debbda0 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:38.451308 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12e31e4-4571-479e-8e0a-124c1debbda0 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:38.456132 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c29401-21da-42ed-b596-90c4a398195b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:38.457300 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c29401-21da-42ed-b596-90c4a398195b admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:38.457572 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c29401-21da-42ed-b596-90c4a398195b admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:38.457835 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c29401-21da-42ed-b596-90c4a398195b admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:38.458540 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c29401-21da-42ed-b596-90c4a398195b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:38.460105 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c29401-21da-42ed-b596-90c4a398195b admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:38.465819 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:44:38.470778 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1431/2900] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 14:44:38.475041 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae726ada-59a7-4918-8d09-00a671077787 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:38.485668 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae726ada-59a7-4918-8d09-00a671077787 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:38.490928 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a85a0d-d2bb-4ba3-904e-be370d0f78df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:38.491587 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a85a0d-d2bb-4ba3-904e-be370d0f78df admin admin] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:38.492076 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a85a0d-d2bb-4ba3-904e-be370d0f78df admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:38.492353 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a85a0d-d2bb-4ba3-904e-be370d0f78df admin admin] PATH_INFO: `/v3/roles` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:38.493081 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a85a0d-d2bb-4ba3-904e-be370d0f78df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:38.494676 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a85a0d-d2bb-4ba3-904e-be370d0f78df admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:38.505877 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1470/2901] 10.222.0.26 () {58 vars in 1122 bytes} [Fri Aug 5 14:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 14:44:38.510063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c29401-21da-42ed-b596-90c4a398195b admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:38.521194 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c29401-21da-42ed-b596-90c4a398195b admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:38.529546 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743dbb9-649a-4124-972a-777e51c5324d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:38.530220 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743dbb9-649a-4124-972a-777e51c5324d admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:38.530494 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743dbb9-649a-4124-972a-777e51c5324d admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:38.530762 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743dbb9-649a-4124-972a-777e51c5324d admin admin] PATH_INFO: `/v3/projects/1394eaae7f134114beb7c0bcdcbc4d17/users/ac3ea8231c4648629a974491822e48f8/roles/29967df108484a30a4c006cde2ae2940` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:38.546704 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2743dbb9-649a-4124-972a-777e51c5324d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1394eaae7f134114beb7c0bcdcbc4d17, user_id=ac3ea8231c4648629a974491822e48f8, role_id=29967df108484a30a4c006cde2ae2940)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:38.549377 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2743dbb9-649a-4124-972a-777e51c5324d admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:38.638560 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1432/2902] 10.222.0.26 () {60 vars in 1483 bytes} [Fri Aug 5 14:44:38 2022] PUT /identity/v3/projects/1394eaae7f134114beb7c0bcdcbc4d17/users/ac3ea8231c4648629a974491822e48f8/roles/29967df108484a30a4c006cde2ae2940 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:44:38.649392 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1e3ef-2858-4cb4-9600-802429298076 None None] REQUEST_METHOD: `POST` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:38.650513 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1e3ef-2858-4cb4-9600-802429298076 None None] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:38.650637 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1e3ef-2858-4cb4-9600-802429298076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:38.678135 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.auth.core [None req-7ac1e3ef-2858-4cb4-9600-802429298076 None None] MFA Rules not processed for user `ac3ea8231c4648629a974491822e48f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80322) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:44:38.679344 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-7ac1e3ef-2858-4cb4-9600-802429298076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:38.742575 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1471/2903] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:44:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:44:38.754647 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743dbb9-649a-4124-972a-777e51c5324d admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:38.756329 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-2743dbb9-649a-4124-972a-777e51c5324d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:38.835501 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743dbb9-649a-4124-972a-777e51c5324d admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:38.840142 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce065bfc-b1ec-4533-bc07-bb1a41c99386 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1ce2a2b4eb405b999d3fb756795176', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:38.840142 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce065bfc-b1ec-4533-bc07-bb1a41c99386 service glance] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:38.840405 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce065bfc-b1ec-4533-bc07-bb1a41c99386 service glance] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:38.840549 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce065bfc-b1ec-4533-bc07-bb1a41c99386 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:38.848768 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce065bfc-b1ec-4533-bc07-bb1a41c99386 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:38.850176 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce065bfc-b1ec-4533-bc07-bb1a41c99386 service glance] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:38.858783 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1433/2904] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:44:38 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:44:48.739699 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=71,keystone:INSERT=3 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:44:48.854858 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=45,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:44:53.707094 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1e3ef-2858-4cb4-9600-802429298076 None None] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:53.709287 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-7ac1e3ef-2858-4cb4-9600-802429298076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:53.812937 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1e3ef-2858-4cb4-9600-802429298076 None None] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:53.820222 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70109e0-49bd-4ba1-959d-4f0744490c15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:53.821031 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70109e0-49bd-4ba1-959d-4f0744490c15 service nova] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:53.821554 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70109e0-49bd-4ba1-959d-4f0744490c15 service nova] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:53.821843 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70109e0-49bd-4ba1-959d-4f0744490c15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:53.823661 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-b70109e0-49bd-4ba1-959d-4f0744490c15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:53.837195 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70109e0-49bd-4ba1-959d-4f0744490c15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:53.838817 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70109e0-49bd-4ba1-959d-4f0744490c15 service nova] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:53.849378 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1472/2905] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:44:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:44:58.044714 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce065bfc-b1ec-4533-bc07-bb1a41c99386 service glance] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:44:58.046054 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ce065bfc-b1ec-4533-bc07-bb1a41c99386 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:58.101386 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce065bfc-b1ec-4533-bc07-bb1a41c99386 service glance] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:44:58.104939 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83668b0a-1f6f-4648-81da-eda3ea3369bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:44:58.105543 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83668b0a-1f6f-4648-81da-eda3ea3369bc service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:44:58.105820 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83668b0a-1f6f-4648-81da-eda3ea3369bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:44:58.106090 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83668b0a-1f6f-4648-81da-eda3ea3369bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:44:58.107809 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-83668b0a-1f6f-4648-81da-eda3ea3369bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:44:58.167526 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83668b0a-1f6f-4648-81da-eda3ea3369bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:44:58.168984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83668b0a-1f6f-4648-81da-eda3ea3369bc service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:44:58.178361 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1434/2906] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:44:58 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:45:03.845021 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=46 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:45:08.174053 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=50 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:45:16.483405 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70109e0-49bd-4ba1-959d-4f0744490c15 service nova] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:45:16.545392 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70109e0-49bd-4ba1-959d-4f0744490c15 service nova] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:45:16.549887 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b644b82-3ae3-4554-90e4-652d34952f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:45:16.550523 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b644b82-3ae3-4554-90e4-652d34952f9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:45:16.550800 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b644b82-3ae3-4554-90e4-652d34952f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:45:16.551067 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b644b82-3ae3-4554-90e4-652d34952f9c admin admin] PATH_INFO: `/v3/users/ac3ea8231c4648629a974491822e48f8` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:45:16.552217 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b644b82-3ae3-4554-90e4-652d34952f9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac3ea8231c4648629a974491822e48f8)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:45:16.554132 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b644b82-3ae3-4554-90e4-652d34952f9c admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:45:16.771483 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-6b644b82-3ae3-4554-90e4-652d34952f9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac3ea8231c4648629a974491822e48f8'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:45:16.778099 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-6b644b82-3ae3-4554-90e4-652d34952f9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac3ea8231c4648629a974491822e48f8'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:45:16.778748 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-6b644b82-3ae3-4554-90e4-652d34952f9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac3ea8231c4648629a974491822e48f8'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:45:16.779348 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-6b644b82-3ae3-4554-90e4-652d34952f9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac3ea8231c4648629a974491822e48f8'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:45:16.791201 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-6b644b82-3ae3-4554-90e4-652d34952f9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac3ea8231c4648629a974491822e48f8'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:45:16.878169 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1473/2907] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:45:16 2022] DELETE /identity/v3/users/ac3ea8231c4648629a974491822e48f8 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:45:16.882713 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79396850-e9fd-4fa3-a183-7555857ac8af None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:45:16.883287 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79396850-e9fd-4fa3-a183-7555857ac8af None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:45:16.883287 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79396850-e9fd-4fa3-a183-7555857ac8af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:45:16.903203 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-79396850-e9fd-4fa3-a183-7555857ac8af None None] MFA Rules not processed for user `5e3fbbd9c18b4be39ebc01f40d1df40f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 14:45:16.910381 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-79396850-e9fd-4fa3-a183-7555857ac8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:45:16.960993 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1435/2908] 10.222.0.26 () {58 vars in 958 bytes} [Fri Aug 5 14:45:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 14:45:16.971456 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b644b82-3ae3-4554-90e4-652d34952f9c admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:45:16.972921 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-6b644b82-3ae3-4554-90e4-652d34952f9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:45:17.029048 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b644b82-3ae3-4554-90e4-652d34952f9c admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:45:17.032541 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8224d0a-31dd-4d98-a958-e017bded3e30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:45:17.033149 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8224d0a-31dd-4d98-a958-e017bded3e30 service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:45:17.033429 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8224d0a-31dd-4d98-a958-e017bded3e30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:45:17.033703 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8224d0a-31dd-4d98-a958-e017bded3e30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:45:17.042636 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8224d0a-31dd-4d98-a958-e017bded3e30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:45:17.044055 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8224d0a-31dd-4d98-a958-e017bded3e30 service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:45:17.053413 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1474/2909] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:45:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:45:17.452058 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79396850-e9fd-4fa3-a183-7555857ac8af None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:45:17.453508 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-79396850-e9fd-4fa3-a183-7555857ac8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:45:17.467642 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79396850-e9fd-4fa3-a183-7555857ac8af None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:45:17.472347 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb653a-a18c-4322-a105-dd4889316d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:45:17.473059 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb653a-a18c-4322-a105-dd4889316d9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:45:17.473343 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb653a-a18c-4322-a105-dd4889316d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:45:17.473642 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb653a-a18c-4322-a105-dd4889316d9d admin admin] PATH_INFO: `/v3/projects/1394eaae7f134114beb7c0bcdcbc4d17` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:45:17.474811 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb653a-a18c-4322-a105-dd4889316d9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1394eaae7f134114beb7c0bcdcbc4d17)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:45:17.476934 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb653a-a18c-4322-a105-dd4889316d9d admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:45:17.576992 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-2aeb653a-a18c-4322-a105-dd4889316d9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1394eaae7f134114beb7c0bcdcbc4d17', 'request_id': 'req-2aeb653a-a18c-4322-a105-dd4889316d9d'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:45:17.583482 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-2aeb653a-a18c-4322-a105-dd4889316d9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1394eaae7f134114beb7c0bcdcbc4d17', 'request_id': 'req-2aeb653a-a18c-4322-a105-dd4889316d9d'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:45:17.678418 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-2aeb653a-a18c-4322-a105-dd4889316d9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1394eaae7f134114beb7c0bcdcbc4d17', 'request_id': 'req-2aeb653a-a18c-4322-a105-dd4889316d9d'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:45:17.689338 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-2aeb653a-a18c-4322-a105-dd4889316d9d admin admin] The token cache is being invalidate because project 1394eaae7f134114beb7c0bcdcbc4d17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:45:17.689620 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-2aeb653a-a18c-4322-a105-dd4889316d9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:45:17.690235 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-2aeb653a-a18c-4322-a105-dd4889316d9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:45:17.691766 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1436/2910] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:45:17 2022] DELETE /identity/v3/projects/1394eaae7f134114beb7c0bcdcbc4d17 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:45:27.049277 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:45:27.686182 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:46:38.355045 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8224d0a-31dd-4d98-a958-e017bded3e30 service neutron] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:46:38.356577 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-c8224d0a-31dd-4d98-a958-e017bded3e30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:46:38.367860 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8224d0a-31dd-4d98-a958-e017bded3e30 service neutron] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:46:38.415399 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35074d9-c158-4a34-81a8-55d0bc0e6af9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:46:38.416003 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35074d9-c158-4a34-81a8-55d0bc0e6af9 service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:46:38.416281 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35074d9-c158-4a34-81a8-55d0bc0e6af9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:46:38.416585 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35074d9-c158-4a34-81a8-55d0bc0e6af9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:46:38.418295 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-a35074d9-c158-4a34-81a8-55d0bc0e6af9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:46:38.427904 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35074d9-c158-4a34-81a8-55d0bc0e6af9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:46:38.429341 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35074d9-c158-4a34-81a8-55d0bc0e6af9 service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:46:38.437163 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1475/2911] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:46:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:46:48.433581 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=26 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:47:16.849421 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb653a-a18c-4322-a105-dd4889316d9d admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:47:16.851452 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-2aeb653a-a18c-4322-a105-dd4889316d9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:47:16.864019 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb653a-a18c-4322-a105-dd4889316d9d admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:47:16.867695 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c624fc5-34a6-4230-9659-2e6815b4ad48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:47:16.868299 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c624fc5-34a6-4230-9659-2e6815b4ad48 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:47:16.868603 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c624fc5-34a6-4230-9659-2e6815b4ad48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:47:16.868878 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c624fc5-34a6-4230-9659-2e6815b4ad48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:47:16.870617 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-0c624fc5-34a6-4230-9659-2e6815b4ad48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:47:16.940990 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c624fc5-34a6-4230-9659-2e6815b4ad48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:47:16.942433 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c624fc5-34a6-4230-9659-2e6815b4ad48 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:47:16.953157 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1437/2912] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:47:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:47:18.137863 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35074d9-c158-4a34-81a8-55d0bc0e6af9 service neutron] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:47:18.146544 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35074d9-c158-4a34-81a8-55d0bc0e6af9 service neutron] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:47:18.150193 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8e540-8b1b-4e58-92ba-8b40f54f46fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:47:18.150793 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8e540-8b1b-4e58-92ba-8b40f54f46fc service neutron] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:47:18.151068 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8e540-8b1b-4e58-92ba-8b40f54f46fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:47:18.151337 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8e540-8b1b-4e58-92ba-8b40f54f46fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:47:18.153090 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-46f8e540-8b1b-4e58-92ba-8b40f54f46fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:47:18.166767 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f8e540-8b1b-4e58-92ba-8b40f54f46fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:47:18.168179 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f8e540-8b1b-4e58-92ba-8b40f54f46fc service neutron] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:47:18.179713 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1476/2913] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:47:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:47:19.236192 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c624fc5-34a6-4230-9659-2e6815b4ad48 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:47:19.237623 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-0c624fc5-34a6-4230-9659-2e6815b4ad48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:47:19.251456 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c624fc5-34a6-4230-9659-2e6815b4ad48 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:47:19.257729 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db74260-e036-42ae-8807-5a7b27952487 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:47:19.258380 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db74260-e036-42ae-8807-5a7b27952487 service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:47:19.258656 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db74260-e036-42ae-8807-5a7b27952487 service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:47:19.258927 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db74260-e036-42ae-8807-5a7b27952487 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:47:19.260716 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-7db74260-e036-42ae-8807-5a7b27952487 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:47:19.273672 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db74260-e036-42ae-8807-5a7b27952487 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:47:19.275503 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db74260-e036-42ae-8807-5a7b27952487 service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:47:19.287090 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1438/2914] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:47:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:47:24.935988 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8e540-8b1b-4e58-92ba-8b40f54f46fc service neutron] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:47:24.937595 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-46f8e540-8b1b-4e58-92ba-8b40f54f46fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:47:24.951551 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8e540-8b1b-4e58-92ba-8b40f54f46fc service neutron] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:47:24.957480 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9d7f2-0599-4ac0-b73a-0223493f66b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ec4940c99fc4fa8a5bc7cdabc4558fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:47:24.957892 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9d7f2-0599-4ac0-b73a-0223493f66b3 service nova] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:47:24.959119 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9d7f2-0599-4ac0-b73a-0223493f66b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:47:24.959119 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9d7f2-0599-4ac0-b73a-0223493f66b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:47:24.961889 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-59e9d7f2-0599-4ac0-b73a-0223493f66b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:47:24.974616 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e9d7f2-0599-4ac0-b73a-0223493f66b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:47:24.976748 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e9d7f2-0599-4ac0-b73a-0223493f66b3 service nova] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:47:24.989234 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1477/2915] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:47:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:47:29.282288 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=40 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:47:30.921933 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db74260-e036-42ae-8807-5a7b27952487 service nova] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:47:30.923411 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-7db74260-e036-42ae-8807-5a7b27952487 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:47:30.935039 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db74260-e036-42ae-8807-5a7b27952487 service nova] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:47:30.938564 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34223904-c0a7-45cd-b24f-3b46ffb74db5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '695e37560f8f495e987d1bd17acc2680', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:47:30.939163 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34223904-c0a7-45cd-b24f-3b46ffb74db5 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:47:30.939439 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34223904-c0a7-45cd-b24f-3b46ffb74db5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:47:30.939708 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34223904-c0a7-45cd-b24f-3b46ffb74db5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:47:30.941556 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-34223904-c0a7-45cd-b24f-3b46ffb74db5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:47:31.005355 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34223904-c0a7-45cd-b24f-3b46ffb74db5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:47:31.007101 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34223904-c0a7-45cd-b24f-3b46ffb74db5 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:47:31.016622 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1439/2916] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:47:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:47:34.407615 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9d7f2-0599-4ac0-b73a-0223493f66b3 service nova] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:47:34.409243 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-59e9d7f2-0599-4ac0-b73a-0223493f66b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:47:34.481386 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9d7f2-0599-4ac0-b73a-0223493f66b3 service nova] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:47:34.486484 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38264ad-d632-4fed-a0f1-3f7a688e60bb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff761f6f81a46098f94bd2ec2be9c1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:47:34.487101 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38264ad-d632-4fed-a0f1-3f7a688e60bb service placement] REQUEST_METHOD: `GET` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:47:34.487382 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38264ad-d632-4fed-a0f1-3f7a688e60bb service placement] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:47:34.487656 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38264ad-d632-4fed-a0f1-3f7a688e60bb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:47:34.489452 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-a38264ad-d632-4fed-a0f1-3f7a688e60bb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:47:34.502556 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38264ad-d632-4fed-a0f1-3f7a688e60bb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:47:34.504125 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38264ad-d632-4fed-a0f1-3f7a688e60bb service placement] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:47:34.514110 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1478/2917] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:47:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:47:37.880539 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34223904-c0a7-45cd-b24f-3b46ffb74db5 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:47:37.882367 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-34223904-c0a7-45cd-b24f-3b46ffb74db5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:47:37.895295 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34223904-c0a7-45cd-b24f-3b46ffb74db5 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:47:37.900692 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc490b96-92b7-4f9e-bb90-a781f31b7e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:47:37.901322 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc490b96-92b7-4f9e-bb90-a781f31b7e67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:47:37.901596 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc490b96-92b7-4f9e-bb90-a781f31b7e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:47:37.901865 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc490b96-92b7-4f9e-bb90-a781f31b7e67 admin admin] PATH_INFO: `/v3/users/f7b9ccd1b8cc4832a9c77513b5f21815` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:47:37.902999 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc490b96-92b7-4f9e-bb90-a781f31b7e67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7b9ccd1b8cc4832a9c77513b5f21815)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:47:37.904978 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc490b96-92b7-4f9e-bb90-a781f31b7e67 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:47:38.099176 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-dc490b96-92b7-4f9e-bb90-a781f31b7e67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7b9ccd1b8cc4832a9c77513b5f21815'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:47:38.111794 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-dc490b96-92b7-4f9e-bb90-a781f31b7e67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7b9ccd1b8cc4832a9c77513b5f21815'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:47:38.194440 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-dc490b96-92b7-4f9e-bb90-a781f31b7e67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7b9ccd1b8cc4832a9c77513b5f21815'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:47:38.195063 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-dc490b96-92b7-4f9e-bb90-a781f31b7e67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7b9ccd1b8cc4832a9c77513b5f21815'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:47:38.202310 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-dc490b96-92b7-4f9e-bb90-a781f31b7e67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7b9ccd1b8cc4832a9c77513b5f21815'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:47:38.208509 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1440/2918] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:47:37 2022] DELETE /identity/v3/users/f7b9ccd1b8cc4832a9c77513b5f21815 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:47:38.441663 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38264ad-d632-4fed-a0f1-3f7a688e60bb service placement] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:47:38.453295 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38264ad-d632-4fed-a0f1-3f7a688e60bb service placement] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:47:38.471756 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=53 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:47:38.511198 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be19abc2-56fb-43c8-92b1-d80fbaad24d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:47:38.511837 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be19abc2-56fb-43c8-92b1-d80fbaad24d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:47:38.512142 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be19abc2-56fb-43c8-92b1-d80fbaad24d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:47:38.512461 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be19abc2-56fb-43c8-92b1-d80fbaad24d0 admin admin] PATH_INFO: `/v3/projects/d408cabccaa44c338d6031955d3a1f7c` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:47:38.513589 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be19abc2-56fb-43c8-92b1-d80fbaad24d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d408cabccaa44c338d6031955d3a1f7c)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:47:38.515516 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be19abc2-56fb-43c8-92b1-d80fbaad24d0 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:47:38.658067 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-be19abc2-56fb-43c8-92b1-d80fbaad24d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd408cabccaa44c338d6031955d3a1f7c', 'request_id': 'req-be19abc2-56fb-43c8-92b1-d80fbaad24d0'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:47:38.665188 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-be19abc2-56fb-43c8-92b1-d80fbaad24d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd408cabccaa44c338d6031955d3a1f7c', 'request_id': 'req-be19abc2-56fb-43c8-92b1-d80fbaad24d0'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:47:38.670545 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-be19abc2-56fb-43c8-92b1-d80fbaad24d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd408cabccaa44c338d6031955d3a1f7c', 'request_id': 'req-be19abc2-56fb-43c8-92b1-d80fbaad24d0'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:47:38.747722 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-be19abc2-56fb-43c8-92b1-d80fbaad24d0 admin admin] The token cache is being invalidate because project d408cabccaa44c338d6031955d3a1f7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:47:38.747939 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-be19abc2-56fb-43c8-92b1-d80fbaad24d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:47:38.748579 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-be19abc2-56fb-43c8-92b1-d80fbaad24d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:47:38.750062 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1479/2919] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:47:38 2022] DELETE /identity/v3/projects/d408cabccaa44c338d6031955d3a1f7c => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:47:38.754112 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc490b96-92b7-4f9e-bb90-a781f31b7e67 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:47:38.755517 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-dc490b96-92b7-4f9e-bb90-a781f31b7e67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:47:38.768796 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc490b96-92b7-4f9e-bb90-a781f31b7e67 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:47:38.824984 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211230c-6c17-4cfd-90df-b46bf0feda93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:47:38.825612 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211230c-6c17-4cfd-90df-b46bf0feda93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:47:38.825889 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211230c-6c17-4cfd-90df-b46bf0feda93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:47:38.826160 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211230c-6c17-4cfd-90df-b46bf0feda93 admin admin] PATH_INFO: `/v3/users/d26da9b0c19544cd95bb818b71225174` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:47:38.827317 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211230c-6c17-4cfd-90df-b46bf0feda93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d26da9b0c19544cd95bb818b71225174)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:47:38.829252 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211230c-6c17-4cfd-90df-b46bf0feda93 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:47:38.835277 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:47:38.990139 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1211230c-6c17-4cfd-90df-b46bf0feda93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd26da9b0c19544cd95bb818b71225174'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:47:39.002215 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1211230c-6c17-4cfd-90df-b46bf0feda93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd26da9b0c19544cd95bb818b71225174'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:47:39.069033 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1211230c-6c17-4cfd-90df-b46bf0feda93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd26da9b0c19544cd95bb818b71225174'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:47:39.069860 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1211230c-6c17-4cfd-90df-b46bf0feda93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd26da9b0c19544cd95bb818b71225174'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:47:39.077458 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1211230c-6c17-4cfd-90df-b46bf0feda93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd26da9b0c19544cd95bb818b71225174'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:47:39.083744 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1441/2920] 10.222.0.26 () {58 vars in 1224 bytes} [Fri Aug 5 14:47:38 2022] DELETE /identity/v3/users/d26da9b0c19544cd95bb818b71225174 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:47:39.545246 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be19abc2-56fb-43c8-92b1-d80fbaad24d0 admin admin] Authenticating user token {{(pid=80322) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:47:39.546929 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.fernet_utils [None req-be19abc2-56fb-43c8-92b1-d80fbaad24d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80322) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:47:39.562159 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be19abc2-56fb-43c8-92b1-d80fbaad24d0 admin admin] Validating token access rules against request {{(pid=80322) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:47:39.623954 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893b645-d9f5-4878-ada2-8c08486cd263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e3fbbd9c18b4be39ebc01f40d1df40f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b40e0244e98a49699194b094ad410d1e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80322) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:47:39.624624 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893b645-d9f5-4878-ada2-8c08486cd263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:47:39.624902 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893b645-d9f5-4878-ada2-8c08486cd263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:47:39.625176 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893b645-d9f5-4878-ada2-8c08486cd263 admin admin] PATH_INFO: `/v3/projects/da68239ae0ec4ebb8e3962bd067f4097` {{(pid=80322) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:47:39.626329 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8893b645-d9f5-4878-ada2-8c08486cd263 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da68239ae0ec4ebb8e3962bd067f4097)` {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:47:39.628225 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8893b645-d9f5-4878-ada2-8c08486cd263 admin admin] RBAC: Authorization granted {{(pid=80322) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:47:39.720092 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-8893b645-d9f5-4878-ada2-8c08486cd263 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da68239ae0ec4ebb8e3962bd067f4097', 'request_id': 'req-8893b645-d9f5-4878-ada2-8c08486cd263'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:47:39.721927 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:47:39.726352 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-8893b645-d9f5-4878-ada2-8c08486cd263 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da68239ae0ec4ebb8e3962bd067f4097', 'request_id': 'req-8893b645-d9f5-4878-ada2-8c08486cd263'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:47:39.731697 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-8893b645-d9f5-4878-ada2-8c08486cd263 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da68239ae0ec4ebb8e3962bd067f4097', 'request_id': 'req-8893b645-d9f5-4878-ada2-8c08486cd263'} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:47:39.828861 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-8893b645-d9f5-4878-ada2-8c08486cd263 admin admin] The token cache is being invalidate because project da68239ae0ec4ebb8e3962bd067f4097 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80322) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 14:47:39.832219 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-8893b645-d9f5-4878-ada2-8c08486cd263 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:47:39.832219 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG keystone.notifications [None req-8893b645-d9f5-4878-ada2-8c08486cd263 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80322) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 14:47:39.832219 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: [pid: 80322|app: 0|req: 1480/2921] 10.222.0.26 () {58 vars in 1233 bytes} [Fri Aug 5 14:47:39 2022] DELETE /identity/v3/projects/da68239ae0ec4ebb8e3962bd067f4097 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 14:47:49.080053 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:47:49.920115 dv-cinder-802955-8-iscsi devstack@keystone.service[80322]: DEBUG dbcounter [-] [80322] Writing DB stats keystone:SELECT=5,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80322) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 14:49:45.624576 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211230c-6c17-4cfd-90df-b46bf0feda93 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 14:49:45.625974 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-1211230c-6c17-4cfd-90df-b46bf0feda93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:49:45.695121 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211230c-6c17-4cfd-90df-b46bf0feda93 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 14:49:45.754291 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41e35eb-206e-4435-a050-e6987cbae012 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fff761f6f81a46098f94bd2ec2be9c1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '319973e011c54b4ca1f2198c8a5e135b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 14:49:45.754902 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e35eb-206e-4435-a050-e6987cbae012 service placement] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 14:49:45.755175 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e35eb-206e-4435-a050-e6987cbae012 service placement] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 14:49:45.755442 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e35eb-206e-4435-a050-e6987cbae012 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 14:49:45.757244 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a41e35eb-206e-4435-a050-e6987cbae012 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 14:49:45.769036 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41e35eb-206e-4435-a050-e6987cbae012 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 14:49:45.770600 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41e35eb-206e-4435-a050-e6987cbae012 service placement] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 14:49:45.787041 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1442/2922] 10.222.0.26 () {60 vars in 1365 bytes} [Fri Aug 5 14:49:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 14:49:55.776216 dv-cinder-802955-8-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=58 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}